Stephen Fry

"Education is the sum of what students teach each other in between lectures and seminars. " - Stephen Fry

Thursday 23 August 2012

Notes on "A Close Examination of Performance and Power Characteristics of 4G LTE Networks"! - Pt1

 These are my notes on "A Close Examination of Performance and Power
Characteristics of 4G LTE Networks" which can be found here

4G is as the name suggests, the forth generation of mobile communication standards. There are currently two competing technologies, Mobile WiMAX and LTE. Long term evolution (LTE) is the technology that this paper considers.

4G Test Android Application

The team designed an android application called 4GTest. It is available in the Play store here. The application measures the network performance including latency, download speed and upload speed.

The application description claim that the results produced by this application are more accurate that any other test using single-threaded TCP tests.     

I tested the application on an old HTC Magic but I couldn't get the application to recognize my internet connection. The application is not compatible with the Nexus 7.  

Overall Results/ Conclusions

The abstract states that "LTE generally has significantly higher downlink and uplink throughput than 3G and even WiFi, with a median value of 13Mbps and 6Mbps, respectively". I am not sure why the median value has been quoted here instead of the mean value (if you know why, please comment below).

The data set for the study was 20 smart phone users over 5 months, this seems like a fairly limited sample size.

The power model of the LTE network, had a less than 6% error rate. It was found that LTE was less power efficient than WiFi and 3G. Testing of android applications highlighted that the device power is now becoming a bottleneck, instead of network performance, compared to 3G.

Long term evolution
LTE aims to enhance the Universal Terrestrial Radio Access Network, more commonly known as 3G. LTE is commercially avaliable but is not yet wide spread. The targeted user throughput is 100Mbps for downlink and 50Mbps for uplink. These targets are distinctly different to median values of 13Mbps and 6Mbps, respectively previously quoted. 

User-plane latency is defined as the one-way transit time between the availability of a packet at the IP layer (the network layer) at the source and the availability of this packet at the IP layer (the network layer) at the destination. This definition means that user-plane latency includes the delay introduced by associated protocols. Since the delay is measured from network layer to network layer, we do not need to consider the delay introduced by the application layer or transport layer.

LTE can be compared to other networks such as Wi-FI and 3G by comparing network statistic such as bit rate, latency, user equipment (UE) power saving etc..

LTE uses Orthogonal Frequency Division Multiplex (OFDM) technology.
OFDM is based on FDM, but FDM wastes bandwidth as you need to leave bandwidth free between different carriers to stop the signals from interfering with each other. OFDM allows the carriers to be more closely spaced together so less bandwidth is wasted.

But OFDM is less power efficient as its more complex and requires linear amplification. To save power, LTE up ink uses a special implementation of OFDM called SC-FDMC.

Discontinuous reception (DRX) is also employed to reduce UE power consumption by some existing mobile technologies. LTE supports DRX. DRX is configured on a per UE basis and allows tradeoff to be made between power saving, delay and signalling overhead.

This study differs from previous ones since it doesn’t use either total on time or simplified LTE. Furthermore it uses real user traces instead of synthetic packets. UMICH refers to the real user data set of 20 smartphone users over 5 months, this data consists of user traces from Wi-Fi and 3G networks but not from LTE networks. So instead Wi-Fi traces are feed into the LTE model simulation framework, Wi-Fi traces were chosen over 3G as the RRT of Wi-Fi is more similar to that of LTE than 3G.

The study shows that LTE is less power efficient than WiFi and 3G for small data transfers but for bulk data transfers LTE is more power efficient than 3G but not Wi-Fi. Because LTE is more efficient than 3G for bulk data transfer then its important to make us of application tools such as Application Resource Optimizer (ARO) (MobiSys11 demo here) in LTE.

LTE is less power efficient than WiFi and 3G, even when DRX was used.

In LTE, the tail timer is the key parameter in determining the trade off between UE energy usage, performance and signalling overhead.

The study identified performance bottlenecks for android applications caused by device processing power, this is detected by monitoring CPU usage of application on the android devices.

Background on Radio Resource Control (RRC) and Discontinuous Reception (DRX) in LTE

Radio Resource Control (RRC) is a signalling protocol between user equipment (UE) and the 3G network (or 4G in this case). Long term evolution (LTE) has two RRC states: connected and idle. The transition from RRC-connected to RRC-idle is made when no data have been received/sent in a time peroid. The transition from RRC-idle to RRC-connection is made when some data is received/sent

Discontinuous Reception (DRX) is a used in mobile communication to conserve power. The UE and the network (in this case LTE) decide on phases when data transfer occurs, outside of these phases the network reciever on the mobile device is turned off thus consuming less energy.

For example. in 802.11 wireless networks, polling is used to control DRX. The mobile device is placed into standby for a set time interval and then a message is sent by the access point to indicate if there is any waiting data, if not then it is placed in standby again.

When LTE is RRC-connected state, then the UE can be in continuous reception, short DRX or long DRX. When LTE is RRC-idle, then the UE is only in DRX mode. Long DRX and short DRX are cycles of the receiver being on and off. The receiver is off for longer in long DRX, which increases delay but reduces emergy consuption. The receiver is on for longer in short DRX, which increases energy consuption but reduces deley. The parameters which dictate the length of time before various transitions, controls the trade-off between battery saving and latency.

Network Measurement
The android application designed to collect test data was called 4GTest and allows the user to switch between 3G, WiFI and LTE. The application made use of M-lab support. Measurement Lab (M-lab) is an distributed server platform for the deployment of internet measurement tools. The server-side tools are open source and the API's allow researchers to develop client tools such as 4GTest.

When considering RRT for LTE, its important to consider the latency of the wired parts of the path from client to server because the latency of LTE is lower than that of 3G so errors (caused by wired parts) become more notable. To minimize the path from client to server, which is not LTE, the nearest server to the client is always used.

If GPS was not available then IP address was used to locate the client. This translation from IP address to GPS is a rough estimate. The service used for this translation was MaxMind. 

To measure RRT and latency variation (difference in latency between connections not packets so its not called Jitter), the application repeatedly established a new TCP connection with the server and measures the delay between SYN and SYN-ACK packet. The median RTT measurements and the variation are reported to central server

To measure peek channel capacity, 4GTest sets up multi-threaded TCP measurements using the 3 nearest servers in M-lab. The throughput test lasts for 20 seconds, the first 5 seconds were ignored due to TCP slow start then the other 15 sec are split into 15 1 sec bins. The average throughput for each bin is calculated and the median of all bins is the measured throughput.

Interestingly, the median is used here instead of average as median reduces the impact of abnormal bins.

To collect packet traces on the android phone, tcpdump was cross-compiled.

To capture CPU usage history, a C program was written to read /proc/stat in the android system.

To understand the impact of packet size on one-way delay, delay was measured for a range of packet sizes and for each packet size, 100 samples were measured.

Power measurements were made using a Monsoon power monitor as power input for the android phone. Phone screen was turned off where possible.

The data set used for analysis is called UMICH and includes full packet traces in TCPdump format including both headers and payloads

The network model simulator takes the binary packet trace files in libpcap format and percessioning is required to collect accurate data.

Application performance 

The sample applications tested where some of the most popular apps. For the browser a simple website and a content-rich website were both tested. The approach taken involved launch the applications and then monitoring upstream, downstream data and CPU usage.

LTE Network characterization

These results are from the public deployment of 4GTest. In the US, the coverage of LTE, WiMAX and WiFi were found by 4GTest to be similar.

The public deployment of 4GTest found that the downlink and uplink throughput were both much higher for LTE than for Wi-FI, WiMAX and 3G technology. High variation in LTE throughput was observed.

For RRT and Jitter, LTE was found to have similar values to WiFi and better values than 3G and WiMAX

One-way delay and impact of packet size
On Wifi, packet size didn’t seem to influence one way delay, in either direction. On LTE, the uplink delay increases as packet size increases.

Modility
The requirements for LTE highlight that the network should be optimized for UE at low mobile speed (0 to 15 km/h) whilst still supporting high mobile speed (15 to 120 km/h). It was observed that in the face of speed changes, RTT remains stable and no significant change in throughput was observed.

Time of Day
Time of day was not found to have significant impact on the observed results .
 


Monday 20 August 2012

TCPtrace - An introduction

What is TCPtrace ?

Wireshark wins over TCPtrace on GUI 


Its a tool designed to analyze the output logs from TCPdump. Previously, in my introduction to TCPdump I highlighted that the output logs created by TCPdump were not plain text and only special programs could interpret them, TCPdump is one of these program, as is Wireshark and TCPtrace.

So TCPtrace takes the output file from TCPdump as an input and it then outputs useful information and graphs.

How do I get TCPtrace ?

I downloaded it from the Ubuntu repositories using the typical 'sudo apt-get install tcptrace'. If this is not possible you can download it from here.

How do I input a TCPdump file to TCPtrace ?

You can call TCPtrace with a TCPdump file using  'tcptrace <my-file>' where my-file is the name of the file outputted by TCPdump. For example you could do something like:
$ sudo tcpdump -v -i wlan0 -w my_tcpdump_output -c 100
$ tcptrace my_tcpdump_output

The above will run TCPdump and create the output file called "my_tcpdump_output", this file is then passed as a argument to the TCPtrace tool

How do i interpret the basic TCPtrace output ?

The structure of the output is (in order from the top) :
  • 1st line returns the name of the TCPdump output file that TCPtrace is currently analyzing
  • Then it printed the version of TCPtrace and then this version was last compiled
  • The next line states the number of packets seen and the number of those which were TCP
  • The following line gives the elapsed wallclock time, this is the time TCPtrace took to process the file and it then gives the average number of packets processed by TCPtrace per second
  • The following line gives the trace file elasped time, this is the time between the capture of the first packet and the last packet in the file
  • The sequential lines contain information on each TCP connection
    • First it gives the address and ports of the two machines involved in the connection
    • Then is the label given to this connection by TCPtrace is printed in parenthesis
    • The number proceeding '>' is the number of packets seen in the first host to second host direction
    • The number proceeding '<' is the number of packets seen in the second host to the first host direction
    • Then the connection is labelled with '(complete)' or '(reset)', with the connection being labelled as complete if SYN and FIN packets were seen
This output is TCPtrace's brief output. Just like TCPdump, you can stop the translation of IP address to domain names using the '-n' opinion.

When using TCPdump, you can see more detailed output using the '-v' option but with TCPtrace you can see more detailed output using the '-l' option.

When adding options to TCPtrace, you need to ensure the you place the extra options before the name of the input file and after the tool name.

When viewing the output from the long mode (when -l is the option) then all information is labelled. I'm now going to explain each label given in long output (warning .. this might take a while):

Packets and ACKS
  • total packets - number of packets sent in that specific direction
  • ack pkts sent - number of ACKs sent in that direction
  • pure acks sent - number of ACK sent without data and the SYN,FIN&RST not set
  • sack pkts sent - number of selective ACKs sent in that direction
  • dsack pkts sent - number of duplicate selective ACKs sent in that direction
  • max sack blks/ack - maximum number of selective ACK blocks seen in any selective ACK packet
Retransmissions
  • unique bytes sent - total number of bytes sent excluding retransmittions
  • actual data pkts - total number of packet with at least 1 byte of TCP payload
  • actual data bytes - total number of bytes seen including retransmittion
  • rexmt data pkts - total number of packets that where retransmittions
  • rexmt data bytes - total number of bytes of data that where retransmittions
Window scaling / Probing
  • zxnd probe pkts - total number of window probe packets seen
  • zxnd probe bytes - total number of bytes sent in window probe packets
  • outoforder pkts - number of packets that arrived out of order
  • pushed data pkts - number of packets with the PUSH bit set, this means that the buffered data should be sent to the receiving application
  • SYN/FIN pkts sent - number of packets with SYN or FIN bits set

etc... (sorry I hate leaving things half done, but I really wanted to move on, its in my to-do list)

How do I get RTT (Round-Trip-Time) from TCPtrace ?

TCPtrace will generate statistics on RRT when using with the opinions '-r' and '-l'. This will give data on RRT including the number of RTT samples found, RTT minimum,RTT maximum, RTT average, RTT standard deviation, RTT from TCP's hand shake. The same data is then available again for full-sized RTT samples only.



Monday 6 August 2012

TCPdump - An introduction

As per usual, if you find any mistakes, concepts that could be explained better or have something to add, then please comment below and I'll correct it.

What is TCPdump ?

TCPdump is a command line tool that is used to look at packets on the network. TCPdump does not generate the packets itself but instead it analyzes the packets generated by other applications and from this, it can determine network behaviour and performance.
Despite being called TCPdump, you can choose from a range of protocols including IP,ARP, TCP and UDP.

How do I get TCPdump ?

From Ubuntu, I simply got it via 'sudo apt-get install tcpdump'. Otherwise the public repo is located here, along side some useful information and the manual pages.
You can also check the dependencies of TCPdump using 'apt-cache depends tcpdump'. The output that this returns for me is:

  Depends: libc6
  Depends: libpcap0.8
  Depends: libssl1.0.0
  Suggests: apparmor
  Conflicts: tcpdump:i386
 
If you get the package for the public repo, then you need will extract the content of the .tar.gz file using 'tar -zxf tcpdump-4.3.0.tar.gz' after using cd to move to the directory that you downloaded the file to, then install the program.

So I've got it.. but how do I use it ?

To start with use 'sudo tcpdump' to output to the terminal, this gives the standard output of TCPdump, using the default arguments since you have not passed TCPdump any arguments yet. To the output will be printed to the terminal, until you stop the program using Ctrl-C.

Don't be scared ... we will now begin the break this output down into understandable section (if you not scared then try 'sudo tcpdump -v' or 'sudo tcpdump -vv'  for a verbose output).

Changing network interface
When I'm using tcpdump, it listened on eth0 by default, but you can change this. Entering 'sudo tcpdump -D' will return a list of network interfaces that you can choose between. For me this returns:
        1.eth0
        2.wlan0
        3.usbmon1 (USB bus number 1)
        4.usbmon2 (USB bus number 2)
        5.any (Pseudo-device that captures on all interfaces)
        6.lo
The first item in the list is the Ethernet port, the 2nd is the Wireless card and the 3,4,5th is self explanatory. The 6th is the loopback virtual interface (more information here on wikipedia)

You can then change the interface, by calling TCPdump using 'sudo tcpdump -i <name-of-interface> '. For most of my work, I use the local Wi-Fi so can change the interface from Ethernet to wireless using 'sudo tcpdump -i wlan0'

Creating a file to store the tcpdump arguments
So far, we have only added one argument when we call TCPdump, but there will be a lot more to come. We can save these arguments, which filter the output of TCPdump to a file and then use the file when we call TCPdump.

Create the file using your favourite text editors (mine in gedit at the moment) and then pass the file to TCP dump using 'sudo tcpdump -F <my-file>'. The file dose not need any special file extension or layout.

You may need to change the file permissions so that TCPdump can extract the file. You can view the file premissions using 'ls -l <my-file>'. The 'ls' part is command line tool to list the files in a directory and the argument -l means use long format so the file permissions will be included. The file 10 characters are the file permissions, they are decoded as follows:
  1. 'd' means this is a directory and '-' means this is a file
  2. 'r' means that the file owner can read the file, '-' means they can't
  3. 'w' means that the file owner can write to the file,  '-' means they can't
  4. 'x'  means that the file owner can execute the file,  '-' means they can't. If this is a directory then 'x' means that the owner can list the files in directory, '-' means they can't.
  5. 'r' means that the file group can read the file, '-' means they can't
  6. 'w' means that the file group can write to the file,  '-' means they can't
  7. 'x'  means that the file group can execute the file,  '-' means they can't. If this is a directory then 'x' means that the owner can list the files in directory, '-' means they can't.
  8. 'r' means that everyone else can read the file, '-' means they can't
  9. 'w' means that everyone else can write to the file,  '-' means they can't
  10. 'x'  means that everyone else can execute the file,  '-' means they can't. If this is a directory then 'x' means that the owner can list the files in directory, '-' means they can't.
For me, the default file permissions are '-rw-rw-r--'. This therefore means that the file owner doesn’t have permission to execute the file. This can be changed using 'chmod u+x <my-file>'. The command line tool is chmod, this is used for changing file permissions, 'u' means we are considering the users permissions, '+' means we are granting a permission and 'x' means we can considering the execute permission. Now, if I redo 'ls -l <my-file>' the new result is -rwxrw-r--.
  
Sending the output to file
You can send the output of tcpdump to a file instead of printing it to the terminal using 'sudo tcpdump -w <my-file>'. This output is not saved as a plain text so you can't just read it in a text editor instead you need to use a special program to read the file and extract information from it. You can do this using tcpdump by 'tcpdump -r <my-file>'. Alternatively you can open it using wireshark, launch wireshark and using File>Open.

 
Filtering information by protocol
To filter packets by protocol, add the name of protocol to the arguments of tcpdump so use something like 'sudo tcpdump <name-of-protocol>'. The main protocols that I'm likely are use are IP, ARP, TCP and UDP but others are available, see the man pages for a full list

Filtering information by direction to different hosts
To filter packets by a direction and host, add the direction and then the host name. Possible direction options are src, dst, src or dst, src and dst. You specify the host using its IP address. You can use the logical operators not, or and and. For example, you can look up the local IP address of your machine using 'ifconfig <name-of-interface>'. If you don't specify the name of the interface, then all interfaces will be listed. Now if you only want to view incoming traffic you can use 'sudo udpdump dst <ip-address>'.

Change the level of detail 
Compared to the level of detail provided by the standard query, The detail can be reduced using '-q' for quiet output or increased using '-v' for verbose, '-vv' for more verbose and '-vvv' for even more verbose.

The opinion '-t' means do not print timestamp on each line and the option '-a' allows you to display each packet in ASCII or '-x' to display in hex or '-X' to display in hex and ASCII
View IP address instead of domain names
You can stop the translation of IP address to domain names using the '-n' opinion and you can stop the translation of port number too using the '-nn' opinion


Sources of Info
Wikipedia article, which contains very little information
Official public repo, including the man pages and FAQ 
Useful online tutorial at openmaniak, this site also has good tutorials on networking tools that I've previously covered including wireshark, OpenVPN, Iperf and ping.

Sunday 5 August 2012

A new week, a new approach

I'm changing my approach from using tools like Iperf and ping to collect network data and then using my java program to analyses the output to writing the scripts for myself and working from the ground up.

It is new territory for me so it really exciting but also a bit daunting. Along side this new work, I now have just 7 days left to prepare for the Google Android Development Camp in London. The next week is looking like its going busy but interesting and by the end I hope to have overcome a steep learning curve.

PLAN FOR MONDAY
  • learn & practice using tcpdump 
  • try using netcat to phase the output of tcpdump
  • work out which language is best for the job in hand
I'll update you again soon...

Wednesday 1 August 2012

Data Collection for Latency, Goodput & Jitter - Demo Pt 5.2

Everyone who I have spoken to about my work since yesterday, has asked me the same question. Why are you writing this in Java ? The answer is that I am going on the Google European Android Development Camp in a few weeks so I am using Java were possible in my work so that I can get familiar with the basics again.

To ensure that the server (my laptop) and the client (my android phone) can address each other I ensure that they are behind the same NAT so that private IP addresses can be used. This ensures that both devices can initialize an connect with the other

My Java code for analyzing and comparing Signpost Diagnostic Application to the results generated by Iperf and Ping, requires the following files:
  • SignOutput.txt - this is generated by the Signpost Diagnostic Application, It can be generated by:
    • Hard coding the server IP address into the application code, found here
    • Uninstall previous versions of the application and connect the android phone via USB
    • Within eclipse, load the application onto the android phone
    • cd into Downloads/android-sdk-linux/platform-tools
    • In another terminal, cd into the location of the demo server and start server using ./server.native
    • use "./adb logcat-s SIGPST >> SignOutput.txt"
    • The location of the file SignOutput.txt is now Downloads/android-sdk-linux/platform-tools, copy the file to /TestingSignpostAppOutput
  • IperfOutput.txt - this is generated by a remote shell on the Android phone, it can be generated by:
    • (In terminal 1) cd into Downloads/android-sdk-linux/platform-tools
    • connect android phone via USB
    • use "./adb shell" to start a remote shell 
    • in another terminal (terminal 2), cd into /TestingSignpostAppOutput so output file is loaded straight into correct directory
    • (In terminal 2) use  iperf -s -u >> IperfOutput.txt
    • (In terminal 1) use iperf -u -c {insert laptop IP}
  • PingDownstreamOutput.txt - this is generated by the server sending pings to the client (the android phone), it can be generated by:
    • cd into  /TestingSignpostAppOutput so output file is loaded straight into correct directory
    • Run a bash script containing the following:
    • #!/bin/bash

      for i in {1..10}
      do
        ping {insert phone IP} -c 10 -n -q >>
      PingDownstreamOutput.txt;
      done
  • PingUpstreamOutput.txt - this is generated by the client (android phone) sending pings to the server, it can be generated by:
    • cd into Downloads/android-sdk-linux/platform-tools
    • Save a bash script (lets call it pingUp) containing the following
    • #!/bin/bash

      for i in {1..10}
      do
        ping {insert laptop IP} -c 10 -n -q >>
      PingUpstreamOutput.txt;
      done
    • Connect android phone via USB and copy script to SD cards using  "./adb push pingUp / sdcard/   "
    •  use "./adb shell" to start a remote shell 
    • In the remote shell cd into /sdcard
    • In the remote shell run the script using ./pingUp
    • exit the remote shell
    • cd into  /TestingSignpostAppOutput
    • copy the PingUpstreamOutput.txt file to the laptop from the android phone using "./adb pull /sdcard/PingUpstreamOutput.txt  

Once you have generated all of these files then you can run my program SignpostOutputAnalysis.java found here which should output the average true and estimated latency, goodput and jitter.


The code in SignpostOutputAnalysis.java is still incomplete and untested, I also have not yet tested my instructions for generating the correct files at the correct locations for the java code to be ran. I will be doing this testing next...