How to Test an Actual Smartphone with Ruby
Hiroki Maruchi and Shoichi Iwai, GBA Co., Ltd.
Table of Contents
1. Introduction
RTK provides the API service called "RTK Thrift API."
This API service provides a feature which accesses RTK resources directly from software. Basically, by just configuring the software, you can rent an actual smartphone with RTK, and you can install apps on that smartphone, run them, and then check the test results.
In this article, a simple user test script will be shown and a sample app that uses RTK Thrift API will be run.
2. RTK Thrift API
RTK provides the API service called "RTK Thrift API."
By using this API service, you can access RTK resources directly from a program. Simply put, you can use RTK by just configuring the software.
What's so good about it then?
If you use this "RTK Thrift API," you can create apps with software, and then test them on actual smartphones. So basically apps can be tested without having use developers, testers, and others.
- Testing can be automated.
- Can be integrated with CI tools like Jenkins.
- Activities to improve testing, such as modifications and refinement of processes, become easier.
What does "testing on an actual device with an API" mean?
In the previous articles, when using RTK, developers and/or testers had to manually run a test on a device with a browser or a button on the RTK app or the like. By using an actual device for testing, a person has to physically touch the device.
However, RTK is a service that provides a device in the cloud. Here, a user does not directly touch the device, but while the RTK is operated with a GUI, operation cannot be automated; this is the final restriction on testing.
This restriction can be eliminated by using the RTK API service, "RTK Thrift API." In short, complicated operations that include operating the device with RTK, the rental of the device before and after that, the capturing of screenshots, installing apps, and so on can all be configured in the program.
For developers and testers who are always battling deadlines and a busy workload, this of course reduces the burden on them. However, more importantly, the test process that up until now could only be performed by certain personnel, can be converted to a script, making the process objective and turning it into an asset.
It is said that the reason why Japanese manufacturing became successful is because the manufacturing process moved away from specialized individuals and with the entire production process in mind, individual processes that are only hypothetical can be constantly improved. However, with current Japanese-style software development, specialization is entrenched and both in terms of process and consciousness moving away from operator-based operations for developers that are subject to results is extremely difficult. In device testing, RTK eliminates the need to physically touch the device during testing. Additionally, this separates the operator from the process and the entire development team can objectively evaluate testing, making it possible to improve the process. Therefore, the productivity of the smartphone development process rapidly increases. Specifically, this contributes greatly to the reduction in man-hours and costs. Furthermore, it can be said that it has the power to fundamentally change manufacturing quality through the entire software development process.
RTK Thrift API Configuration
The following diagram shows the configuration as seen from the perspective of RTK using RTK Thrift API.
Description of the main configuration elements for RTK Thrift API
| Object name | Basic description |
|---|---|
| RTK | A cloud service that provides a smartphone device. RTK is operated with the following methods:
|
| RTK Thrift API | The API interface for RTK. It uses Thrift internally. |
| RTK Thrift server | Server providing the RTK Thrift API feature. Provided by NTT Resonant. Configured by the main service and the device service.
|
| RTK Thrift relay server | A server that relays between the user test scripts and the RTK Thrift server. Part of the program in the PC app and runs on the user's PC; can be downloaded from the RTK Web site. Windows and Mac versions are available. |
| User test script | The program that specifies the test that the RTK user (those reading this article) performs by using a RTK rental device. Can be configured with Java, C, Perl or any programming language that is supported by Thrift. This article uses Ruby. |
RTK consists of a GUI interface and an API interface for the browser and PC app. In the previous articles, for the most part only GUI operations were introduced. With GUI operations, someone has to perform operations manually, like clicking buttons and dialog boxes on the browser and PC app, and dragging to continue with the procedure.
The API is a software-like procedure to run various operations on a GUI with a program. Therefore, if you use this API, you can run the exact same test as you would run on an actual smartphone with just a program. Moreover, the RTK Thrift API has various features to make it easy for a user to use. Therefore, programmers who program tests can leave complicated procedures to the server and library, and all they have to do is program the least number of procedures necessary, so they can focus on the programming of the user test script, which for a user is essential for app testing.
Additionally, RTK Thrift API, as the name suggests, uses the open source code Apache Thrift (https://thrift.apache.org/) internally. In this article, the test script is limited to using Ruby and we will not describe Apache Thrift in detail. If you would like more details about Apache Thrift, please click the link at the end of this article.
Operation procedure for RTK Thrift API
The basic procedure to use RTK Thrift API with software is shown below.
If you are the creator of the user test script, make sure that you first understand the workflow below.
(1) Advance preparation: Apply for an RTK plan
You must apply for a plan on the RTK Web site. You will then receive a login name and password.
The usage restrictions differ depending on the plan. Even if the software a user creates is correct, the user may not be able to run that software due to restrictions in the plan. Furthermore, RTK is currently being actively developed and features may be migrated and/or added in the future depending on sales campaigns and changes to plans. Please make sure to check the plan details and restrictions before running the user test script.
(2) RTK Thrift relay server
Before running the user test script, you need to run the RTK Thrift relay server and have it run in the background on the PC as a daemon. The relay server handles complicated processing of protocols, such as processes commands from the user test script created by the user, grants various requests to the RTK server that exists on the other side of the Internet as seen from the user, and so on, and acts as an intermediary.
The program that is the relay server is in the RTK PC app. Therefore, if the RTK PC app has been installed, it is already on the PC. If you are going to use the RTK with a Web browser only, you must install the PC app. The detailed start procedure for this relay server is shown in the next chapter.
The next part of this document will deal with the configuration of the user test script.
(3) Creating the Thrift API library
In this article, a library for Ruby is prepared, so if you are going to write a script in Ruby, there is no need to compile the .thrift file or set up a library. Additionally, if you use the "rtk_connector" for the Thrift API gem library, you can use the API without having to worry about the Thrift framework. If you use it with other languages, you will need to learn about compiling the .thrift file. If necessary please see the references at the end of this article.
(4) Connecting to the main service
Use the RTK Thrift API main service and log in to RTK.
You will need to prepare the login password that you applied for in advance.
Once you log in, select the device that you are going to use. If necessary, select the rental device, OS, and OS version, and narrow down the device you are going to use.
(5) Connecting to the device service
Select a usable device from the main service and rent it.
The device that you rent is the smartphone that resides in the cloud and on which you perform your test operations and record the results. From the results of this test, you can determine whether the test was successful or not.
If you follow and understand the steps, it shouldn't be difficult to do. "(5) Connecting to the device service" follows the same procedure as testing with a real smartphone. If you collaborate with the tester to create and modify testing, the main discussion will be about (5).
This hereby completes the rough processing flow. Next, let's configure a detailed program and actually try running it.
3. Using Thrift API to automate operation of RTK
Let's try using the RTK Thrift API service and perform an automated test.
Starting the RTK Thrift relay server
Run the following command to start the RTK Thrift relay server.
$ cd /Applications/Remote\ TestKit.app/Contents/Java
$ java -cp remote-device-client-2.0-exewrap.jar -Dconf.directory.company.name=nttr nttr.rtk.MainForThriftServer
2015/06/26 14:56:28| INFO| server.main| Main server started [Main server]
2015/06/26 14:56:28| INFO| server.device| Device server started [Device server]
The RTK Thrift relay server is now running in the background on the PC as a daemon.
Test 1: Capture the browser's screen (test_001.rb)
Let's run the sample test script "test_001.rb."
$ ruby test_004.rb
Once the test script finishes running, the Web screen of the rented smartphone on the RTK cloud is saved on the local user's PC. In the case of this example, a Web screen like the following will be captured.
Explanation of the user test script
The content of the user test script run above will be explained below. If this script is executed from the front and follows the order, it means you have understood the meaning. Below, we categorize the script into segments of their particular meaning and explain what is running. Please compare our explanation with the actual user test script to help you understand the explanation. Once you fully understand what is running, modify the script how you like. Furthermore, the user test script below has been improved. Please also refer to the following:
# Constant list: The user needs to change these constants as necessary.
USER_NAME = "hiroki_maruchi" # RTK login user name.
PASSWORD = "********" # RTK login password.
DIR_RESULT = "/Users/hiroki/work/test" # Work directory. Set in accordance with the result captured from RTK.
This is the login information and work directory.
Change the login information to the login information written on your contract (the script will not run as it is now). Please also change the work directory to match your environment.
# Connect to the Thrift server:
# Connect to the main service
puts "Connecting to the main service..."
socket = Thrift::Socket.new('localhost', ServicePorts::MAIN_SERVICE_PORT)
transport = Thrift::BufferedTransport.new(socket)
transport.open
protocol = Thrift::BinaryProtocol.new(transport)
ms = MainService::Client.new(protocol) # Fetch main service API
# start authentication
puts "Log in..."
ms.login(USER_NAME, PASSWORD) # An exception will occur if login fails.
puts "API authentication successful."
Connect to the main service of the Thrift server. To connect, you need a login name and password, which need to be acquired in advance. If you cannot log in successfully, an exception will occur.
# Fetch device list
devices = client.getDevices
# Narrow down the applicable model from the device list.
device = devices[0] # Select the top device from the device list.
puts "Selected the applicable device!: #{device.productName}/#{device.osType}/#{device.osVersion}"
Obtain the array for the list of devices that can be rented from the main service. Here, the top item in the array is obtained without conditions. (This point is improved in the following script.)
The device name, OS type, and OS version will be displayed for the obtained device.
# Connect to the device server
puts "Connecting to the device server..."
socket = Thrift::Socket.new('localhost', ServicePorts::DEVICE_SERVICE_PORT)
transport = Thrift::BufferedTransport.new(socket)
transport.open
protocol = Thrift::BinaryProtocol.new(transport)
# Start renting the device
puts "Start renting the device..."
ds = DeviceService::Client.new(protocol) # Fetch the device service API
ds.open(device.deviceId) # Rental of the specified device begins
Connect to the device service of the Thrift server. Specify the device that you want to use from the devices that can be provided on the main service. By opening this, you can rent a smartphone on the RTK cloud.
Once you have rented a smartphone, you can operate device objects with various methods in the following way like you would operate a real smartphone.
# Operate the rented device:
puts "Operate the rented device:"
puts ("Capture a Web page:")
ds.captureWebPages(["http://techcrunch.com"], DIR_RESULT) # Save the result of the screen capture to DIR_RESULT.
The above applies the captureWebPages method to the rented device object. This method uses the rented device's browser to display the Web screen and saves that result to the specified path on the user's PC by tapping the displayed screen.
In the user test script, by applying a method (operation) that is similar in meaning to an actual device operation to the object (the rented device), you can run tests that were used on actual devices one at a time.
# End the test.
# Do not explicitly execute close server processing with this program:
# Because it is attached to an end event and executed.
# Modify and if it becomes complicated, consider close processing.
puts "End test."
The program ends here. Normally, if the program is as simple as this example, just ending the program will end the test.
Here, RTK Thrift API actually runs complicated end processing, such as returning the rented device and closing the connection between the device and the server, in the background. In the future, if the test is modified and becomes complicated, the creator of the script may have to explicitly program this procedure. Luckily, in this article the program is not so complicated that you have to worry about it. However, if it becomes necessary, please recall what is written here.
Test 2: Installing the Android app (test_002.rb)
Now, let's run the Android app that the user developed on the rented RTK device.
# Operate the rented device:
puts "Operate the rented device:"
# Install the app
puts " Install the app:"
data = File.binread(FILE_APK)
ds.installAndLaunch(data)
sleep 15 # Standby time when the app starts (desired time).
# Get a screenshot
puts " Get a screenshot:"
id = device.deviceId # Rental model ID
name = device.productName.gsub(/ /, "_") # Name of the rental model. However, convert " " (space) into "_" (underbar).
t = ScreenShotType.new(
fileBaseName: "Capture_#{name}(#{id})",
dirPath: DIR_RESULT
)
ds.captureScreenShot(t);
The parts of the script before and after renting the device are exactly the same as the previous script. In the previous script, the browser in the smartphone was used to capture a Web site screen after renting the device.
In this script, this part is replaced by the script in the box above. Specifically, the Android .apk file created by the user is installed and run on the RTK rented device. Next, the displayed screen is captured.
It's worth noting that the .apk file and the captured image file are on the user PC. The .apk file is installed and started, and the captured screen is output, not on the user PC but rather on the smartphone in the cloud managed by RTK on the other side of the Internet. If there is a bug in the .apk file and the app cannot be run correctly, an exception will occur. An error may be displayed on the image that you capture, which means that a bug has been detected in the program created by the user who ran the test.
Test 3: Installing the Android app (modification; test_004.rb)
The parts related to the RTK Thrift API are compiled in "Ruby gem," the Ruby library feature (rtk_connector).
This can be installed as follows:
$ cd rtk_connector … migrate to gem project directory
$ sudo gem install -l pkg/rtk_connector-0.1.0.gem -V … gem install "rtk_connector"
By converting these parts to gem, you can create a test script focusing on just the essential details of the test when creating a test without having to get involved at all with the comparatively complicated RTK Thrift API configuration. Also through library conversion, reliability increases, repetition speeds up, and a test script can be created easily.
By using the rtk_connector gem created on this occasion, the library related to the RTK Thrift API can be used repeatedly. If the work logic used repeatedly in the same way by the user appears, it may be necessary to convert it to a library.
# Constant list: The user needs to change these constants as necessary.
USER_NAME = "hiroki_maruchi" # RTK login user name.
PASSWORD = "****" # RTK login password.
DIR_RESULT = "/Users/hiroki/work/test" # Work directory. Set in accordance with the result captured from RTK. FILE_APK = "#{DIR_RESULT}/app-debug.apk" # APK file to be uploaded. Before running the file, you must test whether it can be uploaded on the device and RTK.
require 'rtk_connector' # Use rtk_connector gem
begin
# Connect to Thrift server:
# Connect to the main service
puts "RTK API service: Connecting to the main service..."
devices = main_service_connect_and_get_devices(user_name: USER_NAME, password: PASSWORD)
# Connect to device service
puts "RTK API service: Connecting to device service..."
ds = device_service_connect
# Narrow down the applicable model from the device list.
device = select_device(devices)
raise "Failed to narrow down the device list!" if device.nil? || device==false
puts "Select the applicable device!: #{device.productName}/#{device.osType}/#{device.osVersion}"
ds.open(device.deviceId) # Start renting the specified device
# Operate the rented device:
puts "Operate the rented device:"
# Install the app
puts " Install the app:"
data = File.binread(FILE_APK) # Fetch APK (binary data).
ds.installAndLaunch(data) # Install and run APK.
sleep 15 # Standby time when the app starts (desired time).
# Get a screenshot
puts " Get a screenshot:"
id = device.deviceId # Rental model ID
name = device.productName.gsub(/ /, "_") # Name of the rental model. However, convert " " (space) into "_" (underbar).
t = ScreenShotType.new(
fileBaseName: "Capture_#{name}(#{id})",
dirPath: DIR_RESULT
)
ds.captureScreenShot(t);
# End the test.
# Do not explicitly execute close server processing with this program:
# Because it is attached to an end event and executed.
# Modify and if it becomes complicated, consider close processing.
puts "End test."
rescue Thrift::Exception => tx # Thrift exception processing
print 'Thrift::Exception: ', tx.message, "\n"
end
4. Conclusion
In this article, we tested an actual smartphone with ruby using the RTK Thrift API, the API service provided by RTK, and accessed RTK resources directly from a program.
Specifically, we rented a smartphone through RTK, installed an app on the rented device, and tested whether it ran successfully or not. We also modified the script and by converting the shared access part into a Ruby gem library we simplified the programming of the user test script. We used Ruby for the script code and made the script highly readable.
Generally in software testing, as test preparation takes some effort, preparation is simplified and a test script that is easily shared among development team members needs to be prepared. Converting the shared parts to a library and selecting a script language that is highly readable and can be identified easily among members are also important elements. It can be said that Ruby is one appropriate candidate. Furthermore, care should be taken to configure a highly readable script and write proper comments, and review is also necessary to make the test an asset that is repeatable.
References
M. Fewster and D. Graham (1999) Software Test Automation, Addison-Wesley
"Apache Thrift", https://thrift.apache.org
"What is Remote TestKit Thrift API?" https://appkitbox.com/doc/remote-testkit-thrift-api