In this section we discuss the tools available for retrieving data from the nodes, the underlying function for data analysis and system debugging and troubleshooting.
TOSSIM can be used to debug your program, simply by typing 'make pc' during the build process discussed above and using debugging statements. When analyzing the output of the simulation, rather then looking at tuple information, the user has access to the richer printf mechanism. NEED TO PROVIDE ADDITIONAL EXAMPLES HERE
The Data Analysis process is broken up into three steps:
1. Setting up serial forwarders to all the nodes and retrieving a raw byte stream.
2. Converting the byte stream into semantically meaningful tuple data.
3. Providing summarized analysis of the tuple data
We provide the java tools for completing steps one and two. As step three is application-dependent and also customized for each different user, we include some sample tools but envision developers using these as templates for developing their own tools.
Each DSN program, when debugging is enabled (the default setting), writes all new tuple entries to the UART. Consequently, by analyzing this output data, the user can get a fairly detailed representation of the state maintained by each node at any given time and also a chronological progression.
As mentioned initially, Java 1.5 (or higher) is highly recommended for the data analysis portion of DSN development as our tools make heavy use of 'Generics' which are not available in earlier versions of Java. However, we recognize that often that users may wish to deploy on testbeds that are connected to remote servers which may not meet the Java version requirement and where they do not have administrator access in order to be able to upgrade the system. Consequently we provide two different tools to address the different situations. The case where the server meets the requirement can be found directly below and the other case can be found later on, here.
The tools can be found in the code/util directory. The first step is to edit the moteconfig.local
file to suit your specific environment. moteconfig.local.example provides an example template:
serial@COM28:telos
serial@COM29:telos
serial@COM30:telos
This simply tells DSN the locations of the nodes for which it needs to open serial forwarder connections to and retrieve data from. The basic motelist function can be used to determine the port numbers of the connected nodes, and the syntax follows that specified for TinyOS 1.x serial forwarder connections.
Once this file has been created, it is time to open the serial forwarders to all these nodes. This can be done with the following command:
python masssf.py -moteconfig <YOUR CONFIG FILE>
By the same token, we have also implemented an automatic option to just have the tool open serial forwarders to all attached nodes, without the need for a config file. In some cases, this option has not worked as desired and consequently the user could revert to the approach outlined above. The automatic option can be specified as such:
python masssf.py -auto
Serial forwarder connections have now been opened up to all the desired nodes. Two new files have also been created that will be used in the next step.
The first of these is called sfconfig.local and it contains information about the serial
forwarders that were started. As an example, here is what the file could potentially look like:
sf@localhost:7706
sf@localhost:7707
sf@localhost:7708
This information is used by the java program to determine what ports to listen on for data. The second file
can be found in code/gen/.flat_tuplemsg. This file encodes the signature of the different
predicates and tuple types used in the program being analyzed, allowing the byte stream output to be converted
into tuple information. For example, one version of the file could look as such:
linklqi 0 0,2,4
toTransmit 1 0,2
candidate 2 0,2,4,6,8
dest 3 0,2
nextHop 4 0,2,4,6,8
timer 5 0,2,4
...
Understanding the contents of this file is not important as it is automatically generated and should never be edited by you. However, its format is relatively simple. For each line, the first parameter specifies the predicate name, the second parameter specifies the predicate type ID, and the last clause lists all the parameters of the predicate along with the starting point of each parameter in the byte stream.
NOTE: This file is actually created when the DSN program is compiled. Make sure that the version of this file used corresponds to the program you are getting output from, otherwise the tool will give you errors or incorrect output. This is most easily guaranteed by making sure that the last program compiled on the system running the tool is the program downloaded onto the motes.
In order to execute the data analysis tool, the following command should be issued:
java dsntools.Listen2 -sfconfig sfconfig.local -predconfig ../gen/.flat_tuplemsg -v
The format should be relatively self explanatory, except that the final option, '-v', species that not only should the tuple data be outputted, but so should the original byte stream.
The difficulty with this command is that it only outputs data to the screen, and also provides data beyond the basic tuple information. In order to remedy this, we modify the command slightly:
java dsntools.Listen2 -sfconfig sfconfig.local -predconfig ../gen/.flat_tuplemsg -v | grep DisplayThread | tee myOutput
The grep command ensures that only information about the tuples is seen, while 'tee' is a unix command for simulatenously writing output to the screen and to the specified file. Output from the file will look something like this:
12:17:59 [DisplayThread] <11> beacon (65535, 11, 11, 11, 0, 0)
12:17:59 [DisplayThread] <44> linklqi (44, 11, 307)
12:17:59 [DisplayThread] <45> beacon (45, 11, 11, 11, 0, 0)
12:17:59 [DisplayThread] <12> beacon (12, 11, 11, 11, 0, 0)
12:17:59 [DisplayThread] <44> beacon (44, 11, 11, 11, 0, 0)
12:17:59 [DisplayThread] <45> nexthop (45, 11, 11, 420, 1)
12:17:59 [DisplayThread] <25> linklqi (25, 11, 273)
12:17:59 [DisplayThread] <35> linklqi (35, 11, 273)
12:17:59 [DisplayThread] <34> beacon (34, 11, 11, 11, 0, 0)
12:17:59 [DisplayThread] <12> nexthop (12, 11, 11, 189, 1)
Each line contains the information about a tuple, with the name of the predicate and the values of each parameter. The number in brackets indicates the ID of the node at which the tuple is stored (as this is the node that sent the tuple over the UART connection).
In this setup, processing is split up over two different computers. The second system, often your personal machine will need to meet the Java requirement.
The first step is to create a file called sfconfig.local in the code/util directory on the server.
This file stores information about the serial forwarder connection strings, and which nodes to connect to. An
example version of this file is as follows:
serial@/dev/ttyS100:telos
serial@/dev/ttyS101:telos
serial@/dev/ttyS102:telos
...
In this setup, telos motes are connected by USB to the server, and the USB-to-Serial converter assigns port numbers in that fashion. After this has been setup, the serial forwarders can be started and data downloaded. On some systems, starting up a large number of serial forwarders proved problematic and overwhelmed the system. Consequently, this tool uses internal MoteIF structures to grab the data, easing the load on the system.
java dsntools.Listen3 -sfconfig sfconfig.local
This outputs all the data to the screen. The difficulty is additional information such as locks being released, and initial processing information is also included, and this data is not outputted to a file. To remedy this, we modify the command slightly:
java dsntools.Listen3 -sfconfig sfconfig.local | grep "00 00 00" | tee myOutput.txt
Although a bit unorthodox, empirically only the relevant data byte streams contain three bytes of 0's (and they always do in DSN programs), but any suitable grep command could theoretically be used. 'tee' is a unix command that allows output to be simultaneously written to the screen and a designated file.
After enough output has been gathered, this file needs be transferred to the second system. This example
assumes that a code/evaluation directory will hold these files and the tools will continue to be
run from the code/util directory.
Looking at the contents of the file, we just see it is dump of the packet contents in pure byte stream format:
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 41 00 00 41 00 0B 00 BD 00
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 22 00 00 22 00 0B 00 F3 00
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 2C 00 00 2C 00 0B 00 88 0E
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 0C 00 00 0C 00 0B 00 A5 00
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 0D 00 00 0D 00 0B 00 31 02
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 1F 00 00 1F 00 0B 00 F3 00
Thu Apr 12 16:56:29 PDT 2007 09 00 00 00 00 00 7E 00 6C 7D 16 00 00 16 00 0B 00 D8 00
This data now needs to be converted into something more readable. This is done using our second tool:
java dsntools.Listen4 -infile <YOUR DATA FILE> -predconfig ../gen/.flat_tuplemsg | tee myOutput_Processed.txt
Please refer to the previous subsection for a description of the .flat_tuplemsg file and its purpose
This file will convert the byte stream into a more readable format, as such:
Thu Apr 12 12:17:59 PDT 2007 <11> beacon (65535, 11, 11, 11, 0, 0)
Thu Apr 12 12:17:59 PDT 2007 <44> linklqi (44, 11, 307)
Thu Apr 12 12:17:59 PDT 2007 <45> beacon (45, 11, 11, 11, 0, 0)
Thu Apr 12 12:17:59 PDT 2007 <12> beacon (12, 11, 11, 11, 0, 0)
Thu Apr 12 12:17:59 PDT 2007 <44> beacon (44, 11, 11, 11, 0, 0)
Thu Apr 12 12:17:59 PDT 2007 <45> nexthop (45, 11, 11, 420, 1)
Thu Apr 12 12:17:59 PDT 2007 <25> linklqi (25, 11, 273)
Thu Apr 12 12:17:59 PDT 2007 <35> linklqi (35, 11, 273)
Thu Apr 12 12:17:59 PDT 2007 <34> beacon (34, 11, 11, 11, 0, 0)
Thu Apr 12 12:17:59 PDT 2007 <12> nexthop (12, 11, 11, 189, 1)
Just as we stated in the previous subsection, each line represents a single tuple, with the number in brackets indicating the ID of the node at which the tuple is stored, followed by the predicate name and the values of all the parameters.
Now that we have all the tuples, we need a way to use this data. One approach is to use this data to debug the program, generally using the grep command. For example, in our collection application, assuming that our base station is node 0 and that one of the nodes has an ID of 25, I can use the following command to see which messages successfully reached the base station:
grep "store(0, 25" <File with Tuple Data>
Now say that I notice that there is a period during which almost no packets reach the base station. I can then following this up with the command:
grep "<25> nextHop" <File with Tuple Data>
I can then go to that particular time when the packets were lost to see if there was potentially a change in link quality or constant fluctuations in next hops. If this does not identify the problem, I can then use the tuples to trace the missing packets to see where they get lost and identify the problem.
In addition to just debugging, the data can used to provide certain statistics. For example, we used the following tools in order to gather data regarding our MultiHopLQI and Collection DSN programs. We note that these tools were customized for our specific deployment scenarios and are designed to provide templates for your tools, rather than be used straight out of the box. The following commands were used for the tools:
java dsntools.Listen7 -infile <File with Tuple Data> -predconfig ../gen/.flat_tuplemsg | tee myAnalysis.txt
java dsntools.AnalyzeMR -infile <Output of previous command> | tee myStatistics.txt // MultiHopLQI
java dsntools.AnalyzeCollection -infile <Output of previous command> | tee myStatistics.txt // Collection
After this two step process, we get the following output for MultiHopLQI and Collection, respectively (output from the collection and tree routing programs designed earlier in this seciton are what's being analyzed):
Node 12: 0.0
Node 13: 2.0
Node 14: 0.0
Node 15: 1.0
Node 16: 0.0
Node 21: 1.0
Node 22: 1.0
Node 23: 2.0
Node 24: 2.0
Node 25: 2.0
Node 26: 3.0
Node 31: 0.0
Node 32: 1.0
Node 33: 2.0
Node 34: 3.0
Node 35: 2.0
Node 36: 3.0
Node 44: 4.0
Node 45: 3.0
Node 54: 4.0
Node 55: 3.0
Node 56: 4.0
Node 65: 5.0
Node 66: 5.0
Node 74: 5.0
Node 75: 5.0
Node 76: 6.0
0 Hops: 4
1 Hops: 4
2 Hops: 6
3 Hops: 5
4 Hops: 3
5 Hops: 4
6 Hops: 1
[Same Output as MultiHopLQI]
...
Node 12: Transmissions: 0 PRR: 0.1
Node 13: Transmissions: 225 PRR: 1.0
Node 14: Transmissions: 122 PRR: 1.0
Node 15: Transmissions: 547 PRR: 0.225
Node 16: Transmissions: 0 PRR: 0.1
Node 21: Transmissions: 82 PRR: 1.0
Node 22: Transmissions: 89 PRR: 1.0
Node 23: Transmissions: 114 PRR: 0.775
Node 24: Transmissions: 273 PRR: 0.525
Node 25: Transmissions: 69 PRR: 1.0
Node 26: Transmissions: 1 PRR: 0.05
Node 31: Transmissions: 288 PRR: 1.0
Node 32: Transmissions: 57 PRR: 0.325
Node 33: Transmissions: 107 PRR: 0.725
Node 34: Transmissions: 86 PRR: 0.225
Node 35: Transmissions: 528 PRR: 0.075
Node 36: Transmissions: 265 PRR: 0.025
Node 44: Transmissions: 1318 PRR: 0.5
Node 45: Transmissions: 40 PRR: 0.35
Node 54: Transmissions: 371 PRR: 0.05
Node 55: Transmissions: 91 PRR: 0.7
Node 56: Transmissions: 0 PRR: 0.025
Node 65: Transmissions: 146 PRR: 0.0
Node 66: Transmissions: 0 PRR: 0.0
Node 74: Transmissions: 254 PRR: 0.05
Node 75: Transmissions: 88 PRR: 0.2
Node 76: Transmissions: 343 PRR: 0.075
Total Transmissions: 5504
Overall PRR (Additive): 11.1
The MultiHopLQI analysis tool shows the average number of hops each node was from the base station during the observed period, and then puts the nodes into buckets to build a frequency diagram. The Collection analysis tool builds on this by providing the packet reception rate at the base station for each node sending data, and also an overall end-to-end reliability percentage.
These are just example statistics. This section should have provided the fundamental toolkit for buildling customized data analysis tools.