Figure 1: Abstract-Lite client interface
Abstract-Lite offers an interactive interface to visualize your agents' activity traces. Abstract-Lite is a simpler version of ABSTRACT that does not use ontologies nor SPARQL queries. Abstract-Lite rather relies on XPATH instructions and XSL stylesheets to help you customize your visualizations. Abstract-Lite can display the traces generated by your agents in real time.
Section 1 (Overview) introduces Abstract-Lite's general features. Section 2 (Customize the trace visualization) explains how to interactively customize the visualization through the customization panels. Section 2 assumes that you have basic knowledge of XPATH. Section 3 (Customization with XSL Stylesheets) introduces the usage of stylesheets for a more advanced customization. Section 3 assumes that you have basic knowledge of XML and XSL. Section 4 (Generate the trace in real time) explains how to implement the tracing functions in a Java agent.
Abstract-Lite has a client/server architecture. You can either install your own Abstract-Lite server by following the installation procedure, or use our Abstract-Lite Server hosted by the Liris Lab. The client side has been successfully tested with recent versions of Firefox and Chrome, but does not yet support other browsers.
Open the Abstract-Lite home page with Firefox or Chrome.
Click Register to create a new user.
Log in with the user you have created.
You obtain the page in Figure 1.
Figure 1: Abstract-Lite client interface
1. The Menu Panel (top of the page) provides administrative commands.
2. The Trace List Panel (right side of the page) lets you select a trace from the list. The trace list is initialized with a demo trace when your account is created. The trace list is further populated when you upload new traces or when your agent generates new traces via streaming. Check the Always play last checkbox to automatically display the last trace in the list.
3. The Customization Panel (central column) lets you manage your different customizations that specify how your traces are visualized. A customization is a set of visualization rules. In this panel, the Save customization as form lets you name your customizations and save them. The Customization List (empty here) lets you reload a customization.
4. The Visualization Rules Panel (column on the left) lets you define your visualization rules. A visualization rule is a rule that defines a graphical symbol to represent an aspect of an event. In this panel, the Instantaneous Symbol form and the Interval Symbol form let you create new visualization rules as explained in Section 2.
Click on the demo trace in the trace list. This makes the player appear. In the player, click on the Play button to start playing the trace. You obtain the demo trace visualization in Figure 2.
Figure 2: The trace player
Click the Pause button to stop playing the trace.
Click the plus and minus buttons to zoom in and out over the horizontal axis.
Click the P button to open a printable copy of the trace in a new window. Use your browser's print function to print the trace from this new window.
You can drag the trace to the left or to the right with your mouse by clicking/dragging within the player area.
You can click on a symbol to display the data attached to the event related to this symbol. The attached data will appear in a new box just below the player. This box is called the Event Data box.
You define the visualization rules through the Instantaneous Symbol form or the Interval Symbol form in the visualization rule panel. Note that you can only access these forms while the trace is playing, and the changes take effect immediatly during playing. Each of the input fields in these forms takes an XPATH expression. Abstract-Lite interprets the XPATH expressions in the context of each event in the trace. Each event contains an XML tree that you can freely define when generating the trace. Please have a look at the XML structure of our demo trace by downloading the demo trace (using the downward arrow next to the identifier of the trace in the trace list).
Abstract-Lite applies the visualization rules to each event and interprets the result to generate graphical symbols in the player. The field Condition determines whether the symbol is drawn or not.
On creation, your account was populated with a set of four demo visualization rules shown in Figure 3.
Figure 3: The demo visualization rules
Click on ⎘ (next to the rule identifier) to copy the fields of a specific rule to the symbol form. This feature is used to edit a rule. To modify a rule, just edit it, modify it, save the new version, and delete the previous version from the list.
Click on ✗ to delete a rule.
A new rule is saved and added to the list when you click on the Send button in the symbol form. The player is also updated to reflect the new set of rules.
In our demo trace, the events contain a field named primitive_enacted_schema. This field indicates the action that the agent has just done ('^': turn left, 'v': turn right, '>': move forward). The first demo rule is responsible for drawing a green triangle pointing upward for each turn left event:
Figure 4: Editing the visualization rule of turn left events.
This rule specifies that the symbol 'up' (a triangle pointing up) should be drawn for each event whose primitive_enacted_schema field is equal to the string '^'. This symbol should be drawn in green ('00EE00') and located 10 pixels above the centre of the player area.
Similarly, the visualization rule in Figure 5 displays a green triangle pointing downward for each turn right event. Figure 5 also shows the tooltip window that lists the different shapes available by default.
Figure 5: Editing the visualization rule of turn right events.
The visualization rule in Figure 6 displays a blue triangle pointing frontward for each move forward event:
Figure 6: Editing the visualization rule of move forward events.
Note that despite the term instantaneous, instantaneous symbols have a width that can be used to represent the event's duration (provided that the event's duration was recorded in your trace). To do so, customize the X scale field with an expression that evaluates as the event's duration using the information available in your trace.
An interval symbol is a symbol (e.g., a line) that represents the interval between a begin event and an end event.
The Interval Symbol form works similarly to the Instantaneous Symbol form. For example, the fourth rule draws the lines that represent the interval during which the property 'current_observation/color' remains unchanged. These lines are positioned vertically according to the property 'current_observation/distance':
Figure 7: Editing the visualization rule of an interval symbol.
In the field End condition of this example, the variable $last represents the previous event in the trace (standard XPATH instruction).
To facilitate the specification of the visualization rules, we have defined the additional function my:ifthenelse.
my:ifthenelse(<condition>, <value if true>, <value if false>)
If the first parameter is true, this function returns the value of the second parameter. If the first parameter is false, this function returns the value of the third parameter.
Note that you can specify that an interval symbol will change its appearance depending on its ending condition. This is useful when the trace is displayed in real time. For example, an interval symbol can be displayed in a blue color as long as it is not yet ended, and turn red or green depending on how it ends.
You can now upload your own trace and start customizing its visualization (using "Upload a trace" in the Trace list).
Your trace file should at least have the tags <trace id=""> <slice date = ""> <event date="" source=""> and <clock>. The events with the same date should be grouped within the same tag <slice>. Table 1 gives an example of a minimal file.
Table 1: Example trace file.
<trace id="1"> <slice date = "1"> <event date="1" source="source1"> <clock>1</clock> <type>foo1</type> </event> <event date="1" source="source2"> <clock>1</clock> <type>foo1</type> </event> </slice> <slice date = "2"> <event date="2" source="source1"> <clock>2</clock> <type>foo2</type> </event> </slice> </trace>
XSL stylesheets offer a more flexible way of customizing your visualization than the visualization rules presented in Section 2. By customizing your XML stylesheets, you have total freedom to specify how your trace is visualized.
We, however, had to disable this feature by default because such freedom could open a security flaw on the server. You can either use this feature on your own server, or don't hesitate to ask Olivier and Pierre-Yves for being granted this feature on the Liris server.
Once you are granted this feature, you see the Stylesheet panel in Figure 8. The Stylesheet panel lets you modify, download, and upload the stylesheets.
Figure 8: The stylesheet panel.
This section introduces the pipeline structure of Abstract-Lite and presents how to create customized XSL stylesheets within this structure.
Abstract-Lite processes observed elements (named obsels) through a transformation pipeline. At the pipeline's input, an obsel is created from each event in the trace. The pipeline then processes the obsels through a series of transformation components to progressively construct the SVG code (Scalable Vector Graphics) that specifies how the obsel must be displayed. At the pipeline's output, an obsel thus corresponds to a graphical symbol to be drawn in the player.
The pipeline structure is defined in the file pipeline.xml. To prevent loops, transformation components are organized into levels: the levels that appear first in the file apply first.
Each transformation component has one or several source properties that define the input. There are two special sources: __input__ that is the initial source of raw events, and __config__ that defines events coming from the user interface. The output element at the end of the pipeline file defines the output of the pipeline.
Table 2: The default file pipeline.xml.
<pipeline> <default-scale>6.666666</default-scale> <default-center>0</default-center> <graph-width>240</graph-width> <levels> <level> <transformation type="XSLT"> <name>select-normalize</name> <sources> <source>__input__</source> </sources> <file>select-normalize.xsl</file> </transformation> <transformation type="XSLT"> <name>clock</name> <sources> <source>__input__</source> </sources> <file>clock.xsl</file> </transformation> </level> <level> <transformation type="PHP"> <name>buffer-reconfigure</name> <sources> <source>__config__</source> <source>select-normalize</source> </sources> <file>BufferReconfigureTransformation.inc.php</file> <classname>BufferReconfigureTransformation</classname> </transformation> <transformation type="XSLT"> <name>tooltip</name> <sources> <source>select-normalize</source> </sources> <file>tooltip.xsl</file> </transformation> </level> <level> <transformation type="XSLT"> <name>config-SVG</name> <sources> <source>select-normalize</source> <source>buffer-reconfigure</source> </sources> <file>configSVG.xsl</file> </transformation> <transformation type="XSLT"> <name>simple-SVG</name> <sources> <source>select-normalize</source> </sources> <file>simpleSVG.xsl</file> </transformation> </level> </levels> <output> <source>simple-SVG</source> <source>config-SVG</source> <source>clock</source> <source>tooltip</source> </output> </pipeline>
Table 3 summarizes the main stylesheets provided by default.
Table 3: The main stylesheets provided by default.
| Stylesheet | Description | Remarks |
|---|---|---|
| pipeline.xml | The pipeline defines the order in which the stylesheets are processed. | <graph-width> defines the height of the trace player area. |
| clock.xsl | Compute the basic information needed by the player. | |
| simpleSVG.xsl | Draw the various tapes in the trace. | The visual field. |
| BufferReconfigureTransformation.php | Call the visualization rules. | |
| configSVG.xsl | Generate the graphical symbols. | |
| select-normalise.xsl | Can be used to filter the incoming events | By default, it doesn't filter anything out. |
| tooltip.xsl | Define the format of the data in the Event Data box. | |
| shapes.xsl | Define the shapes of the symbols. | The time-stamps |
The pipeline generates instructions that are passed to the player. The player then executes these instructions to generate the display. There are five instructions: add, delete, clock, tooltip, and save.
The instruction add adds a given symbol (i.e., an SVG element) to the player.
The instruction delete deletes a symbol from the player.
The instruction clock shifts the player to a specific timestamp. (So far, each new event triggers a clock instruction to synchronise the player. We, however, can imagine some types of events that would not synchronize the player (anticipated events)).
The instruction tooltip sets the association between a symbol and the event displayed in the Event Data box.
The save instruction saves some data (a value or a node-set) for use in subsequent processings. Specifically, the save instruction is used to compute interval symbols from multiple events.
The next section illustrates the use of these instructions with examples.
Table 4 shows the demo simpleXML.xsl stylesheet. Notice the include instructions: in particular, shapes.xsl provides shape-drawing facilities in the form of the draw-shape template.
Table 4: The default simpleXML.xsl stylesheet.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:exsl="http://exslt.org/common" xmlns:math="http://exslt.org/math"
xmlns:dyn="http://exslt.org/dynamic" extension-element-prefixes="exsl math dyn"
version="1.1">
<xsl:param name="state-filename" />
<xsl:param name="name" />
<xsl:output method="xml" omit-xml-declaration="yes" indent="no" />
<xsl:strip-space elements="*" />
<xsl:include href="obsel-state.xsl" />
<xsl:include href="shapes.xsl" />
<!-- Match the obsels from the select-normalizer, that is, the ponctual
obsels. -->
<xsl:template match="obsel" mode="named-saves">
<xsl:param name="state" />
<!-- We first define some variables which values we will use to draw the
shape. -->
<xsl:variable name="obsel-type">
<xsl:choose>
<xsl:when test="type">
<xsl:value-of select="type" />
</xsl:when>
<xsl:otherwise>
<xsl:text>action</xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<!-- Lateral position of the shape. -->
<xsl:variable name="begin-position" select="0" />
<xsl:variable name="end-position"
select="$begin-position + number(@end) - number(@begin)" />
<xsl:choose>
<xsl:when test="$obsel-type = 'action'">
<add>
<!-- We create the embedding group. -->
<g id="{@id}-ns" obsel-id="{@id}" date="{@date}" stroke="#000000">
<xsl:call-template name="draw-num-iter">
<xsl:with-param name="begin-position" select="$begin-position" />
<xsl:with-param name="end-position" select="$end-position" />
<xsl:with-param name="vertical-offset" select="-110" />
</xsl:call-template>
</g>
</add>
</xsl:when>
<xsl:otherwise></xsl:otherwise>
</xsl:choose>
</xsl:template>
<xsl:template name="draw-num-iter">
<xsl:param name="begin-position" />
<xsl:param name="end-position" />
<xsl:param name="vertical-offset" />
<xsl:if test="number(@date) mod 10 = 0">
<xsl:call-template name="draw-shape">
<xsl:with-param name="vert-level" select="$vertical-offset" />
<xsl:with-param name="begin-position" select="$begin-position" />
<xsl:with-param name="shape-type" select="'tick-sided-left'" />
<xsl:with-param name="text-value" select="clock" />
<xsl:with-param name="shape-color" select="'#000000'" />
<xsl:with-param name="height" select="'280'" />
</xsl:call-template>
</xsl:if>
</xsl:template>
</xsl:transform>
The input is defined by the template that matches obsel elements. This template applies once for each obsel, and calls sub-templates to actually generate the various symbols. Only one sub-template is called at the moment: the draw-num-iter template. This template is the one that draws the timestamp ticks.
If you look at what this template does in more details, you will notice that the SVG elements (including those returned by the draw-shape template) are embedded into an <add> element. This is how the add instruction works: everything inside an <add> element will be added to the player.
Just inside this add element, you can see a <g> element with several attributes. These attributes specify how the player handles the symbol and places it in the display area. The id element is necessary if you want to reference the symbol later (for example to delete it). The date attribute (timestamp) will determine where the symbol is placed on the x axis. In this example, the obsel-id corresponds to the id of the original event. The id will be used (only) to find the corresponding event when the user clicks on this symbol in order to display the event's data in the Event Data box. (You must be careful to format this data with the corresponding ids; our policy here is to use the original event ids). The stroke attribute is not necessary, just standard SVG.
In the case of interval symbols, the attributes begin and end must be used instead of the attribute date. The player will stretch the interval symbol progressively until it ends (an interval symbol is one-pixel wide before stretching).
Inside this <g> element, we find the template that will generate the effective SVG code to display. To generate this code, the draw-num-iter template calls draw-shape. The parameters taken by the draw-shape template are listed in Table 5.
Table 5: The draw-shape parameters.
<xsl:template name="draw-shape"> <xsl:param name="vert-level" /> <xsl:param name="begin-position" /> <xsl:param name="shape-type" /> <xsl:param name="image-url" select="''" /> <xsl:param name="text-value" select="''" /> <xsl:param name="shape-color" select="''" /> <xsl:param name="height" select="'10'" /> <xsl:param name="transform" select="''" /> ... </xsl:template>
These parameters mean:
Now, let us come back to the example of the interval symbol used in Section 2. We want to draw an interval symbol that represents the duration during which the field current_observation/color remains unchanged. For each incoming obsel, we must test if the value of current_observation/color is the same as in the previous obsel. Then, we must expand the previous obsel up to the date of the current obsel. If current_observation/color has changed, we must finish the previous obsel and create a new one.
The first thing is to make our own drawing template and call it from the entry point. The call instruction is shown in Table 6
Table 6: Calling the drawing template.
<xsl:template match="obsel" mode="named-saves">
<xsl:param name="state" />
<!-- We first define some variables whose values we will use to draw the
shape. -->
<xsl:variable name="obsel-type">
<xsl:choose>
<xsl:when test="type">
<xsl:value-of select="type" />
</xsl:when>
<xsl:otherwise>
<xsl:text>action</xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<!-- Lateral position of the shape. -->
<xsl:variable name="begin-position" select="0" />
<xsl:variable name="end-position"
select="$begin-position + number(@end) - number(@begin)" />
<xsl:choose>
<xsl:when test="$obsel-type = 'action'">
<!-- This, here :. -->
<xsl:call-template name="draw-example-symbol">
<xsl:with-param name="state" select="$state" />
</xsl:call-template>
<!-- Just up there. -->
<add>
<!-- We create the embedding group. -->
<g id="{@id}-ns" obsel-id="{@id}" date="{@date}" stroke="#000000">
<xsl:call-template name="draw-num-iter">
<xsl:with-param name="begin-position" select="$begin-position" />
<xsl:with-param name="vertical-offset" select="-110" />
</xsl:call-template>
</g>
</add>
</xsl:when>
<xsl:otherwise></xsl:otherwise>
</xsl:choose>
</xsl:template>
The drawing template is shown in Table 7.
Table 7: Exemple drawing template of an interval symbol.
<xsl:template name="draw-example-symbol">
<xsl:param name="state" />
<xsl:variable name="sym-state"
select="$state/save[@name='example-sym']/symbol" />
<xsl:variable name="last-state" select="$state/save[@name='example-last']" />
<xsl:choose>
<!-- In the case we have to create a new symbol: at the beginning or
when colors mismatch between this element and the previous one. -->
<xsl:when
test="not($last-state/color) or $last-state/color != current_observation/color">
<!-- All we have to do is to add the symbol. -->
<add>
<g id="{@id}-s" obsel-id="{@id}" date="{@date}" begin="{@date}"
end="{@date}" stroke="#000000">
<xsl:call-template name="draw-shape">
<xsl:with-param name="vert-level"
select="- current_observation/distance div 5 + 75" />
<xsl:with-param name="begin-position" select="0" />
<xsl:with-param name="shape-type" select="'line'" />
<xsl:with-param name="shape-color"
select="concat('#', current_observation/color)" />
</xsl:call-template>
</g>
</add>
<!-- We save the symbole caracteristics, we will probably have to draw
it again. -->
<save name='example-sym'>
<symbol id="{@id}-s" obsel-id="{@id}" begin="{@date}" end="{@date}"
vert-level="{- current_observation/distance div 5 + 75}"
color="#{current_observation/color}" />
</save>
</xsl:when>
<xsl:otherwise>
<!-- We want to lengthen the symbole, so we first have to delete
the old one, before we draw a longer one -->
<delete obsel-id="{$sym-state/@id}" />
<!-- We draw the longer symbol, using the saved properties. -->
<add>
<g id="{$sym-state/@id}" obsel-id="{$sym-state/@obsel-id}" date="{$sym-state/@begin}"
begin="{$sym-state/@begin}" end="{@date}" stroke="#000000">
<xsl:call-template name="draw-shape">
<xsl:with-param name="vert-level" select="$sym-state/@vert-level" />
<xsl:with-param name="begin-position" select="0" />
<xsl:with-param name="shape-type" select="'line'" />
<xsl:with-param name="shape-color" select="$sym-state/@color" />
</xsl:call-template>
</g>
</add>
</xsl:otherwise>
</xsl:choose>
<!-- We save the current obsel color for the next iteration. -->
<save name='example-last'>
<color>
<xsl:value-of select='current_observation/color' />
</color>
</save>
</xsl:template>
The nodes are saved using the <save> instruction, and retrieved through the state param the next time the template is executed. This allows us to remember the last color value thanks to the example-last save, and the symbol characteristics thanks to the example-sym save.
This section describes how to implement a java agent that sends traces to Abstract-Lite via streaming. We apologize for the absence of facilities for other programming languages than Java. If you wish to use Abstract-Lite with other kinds of agents, please contact us.
Click My cookie in the main menu to obtain your private cookie value associated with your account. Your agent will use this cookie value to stream its traces to your account in Abstract-Lite.
Download the Abstract-Lite java package by following the installation procedure. This package provides the java tracer class that your agent can use to send events to Abstract-Lite. This java tracer class is located in the subdir java-tracer/. A javadoc is available in java-tracer/tracer-doc/index.html and online.
Your agent must first instantiate the tracer of type XMLStreamTracer with two parameters. The first parameter is the URL to which the events must be sent. The second parameter is the cookie value. The instantiation instruction looks like this:
m_tracer = new XMLStreamTracer("http://vm.liris.cnrs.fr:34080/abstract/lite/php/stream/", "fjSmkmyvAKgByfDAfXuYGjAJLzrWrf");
Your agent can then generate events using the following methods:
(a) Generate an event with the startNewEvent method:
m_tracer.startNewEvent(timestamp);
(b) Populate the event with the addEventElement method:
Object element = m_tracer.addEventElement("label", "textValue");
(c) If necessary, add any number of element sub-levels with the addSubelement method:
Object subElement = m_tracer.addSubelement(element, "label", "textValue");
(d) When the event is completed, the agent can finish it with the finishEvent method. Otherwise, an event will be automatically finished when the agent starts the next event:
m_tracer.finishEvent();
The agent can signal the end of the trace with the close method.
We recommend creating only one event for each timestamp, to facilitate the specification of the visualization rules.
To refer to the ABSTRACT project, please use the following reference:
Supporting Activity Modeling from Activity Traces. Olivier L. Georgeon, Alain Mille, Thierry Bellet, Benoit Mathern, Frank E. Ritter (2011). Expert Systems. doi: 10.1111/j.1468-0394.2011.00584.x.
To refer to ABSTRACT-LITE and its usage to analyse the activity of artificial agents, please use the following reference:
Early-Stage Vision of Composite Scenes for Spatial Learning and Navigation. In the proceedings of the First Joint IEEE Conference on Development and Learning and on Epigenetic Robotics (ICDL-EPIROB 2011). Olivier L. Georgeon, James B. Marshall, Pierre-Yves R. Ronot. Frankfurt (24-27 August 2011).
Abstract-Lite was developed by Pierre-Yves Ronot at the Liris lab in Lyon (France), as part of the ABSTRACT project and of the IDEAL project.
For any question, comment, feedback, congratulation message, or proposition of collaboration, please contact Pierre-Yves Ronot and Olivier Georgeon.