204 views
openEASE Hands-On ===== For questions please contact Sascha Jongebloed (jongebloed@uni-bremen.de) Visit openEASE -------------- To test openEASE visit the website https://data.open-ease.org/. You need to register and login to openEASE to try it. We recommend to try the NEEM "Popcorn-Demo-In-Projection" by Alina Hawkin from 12/25/22 (https://data.open-ease.org/QA?neem_id=6357fad897fdbc7c4d98ccbc). Example queries --------------- Queries are written in a Prolog-like language. For example to get the instance of the PR2 robot in the experiment you can use: ``` has_type(Robot, knowrob:'PR2') ``` You send the query with Ctrl + Enter. Please note that you can use "Next solution" if there are more valid bindings for the query. The data of the NEEM's are stored in a knowledge graph. To get a relation in the knowledge graph you can always use the triple predicate. For example to get the relations of the PR2 use ``` triple(knowrob:'PR2',P,O) ``` where P is the relation and O is the entity to which the PR2 is holding the relation P. To see all potential relations you can look into SOMA (see [Further Information](https://hackmd.informatik.uni-bremen.de/51GdGvG3Q0Wnkh_h_Uzsag#Further-information)) You can have a look at the example queries. We recommend as the first query to test, the query for the longest event. You can find the query under Examples -> Occurrences -> Duration. If you just want to copy and past the query you will find the query below: ``` findall([Duration, Evt], ( event_interval(Evt, Begin, End), number(End), Duration is End - Begin ), Durations), max_member([MaxDuration, LongestEvt], Durations) ``` If you did run this query (you possibly have to wait a bit for all visualizations), you can see a replay of the event and a timeline. If you want to try follow-up queries you could click on one of the events in the timeline and choose a follow-up query on the left (e.g. "What is this?" to see an overview of the information). A query to ask for an object that is participating in an event can be written like this: ``` % Get the bowl rdf_db:rdf_equal(Obj, soma:'IkeaBowlWw1'), % Get an event where the object participates in has_participant(Evt, Obj) ``` The last query in this section is an extension of the query above. We will now also ask for the position of the object at the begin of the event: ``` % Get a fetching event executes_task(Evt, Task), has_type(Task, soma:'Placing'), % Get the Gripper rdf_db:rdf_equal(Obj, 'http://knowrob.org/kb/PR2.owl#PR2Gripper_0_L'), % Get the location of the object at the start of the event % Get the start and end time event_interval(Evt, Start, End), % Defined a timescope at beginning time_scope(Start, Start, QScope), tf:tf_get_pose(Obj, [map, Pose, Rotation], QScope,_) ``` Complex query ------------- You can combine the predicates we introduced to create more complex queries. This query asks for a successfull fetching action of the PR2 in which an object is manipulated. ``` % Get an action that is described by a task executes_task(Action, Task), % The task should have has_type(Task, soma:'Placing'), % The action should have been successfull action_succeeded(Action), % Get a participant of the action has_participant(Action, Participant), % The participant is working with the PR2 has_type(Participant, knowrob:'PR2'), % Get another participant of the action has_participant(Action, Object), % This participant should have the role of item has_role(Object, ItemRole) during Action, has_type(ItemRole,soma:'Item') ``` Further information ------------------- We provide some tutorials on openEASE (choose the Tutorial tab on the website). For finding the list of possible predicates, you can have a look at the KnowRob documentation https://knowrob.github.io/knowrob/master/ For documentation on SOMA visit https://ease-crc.github.io/soma/. To search the ontology we recommend a tool like Protege https://protege.stanford.edu/. If you want to use Protege (with less features) in the web, you can use WebProtege (hosted by Standford). The link to our ontology is https://webprotege.stanford.edu/#projects/1423ae6c-c223-4098-b572-36dc71a7b8db/edit/Classes . You potentially need to reclick on the link after the login. Exercises ---------- In this exercise, you will write queries for a recorded robotic NEEM. Under the following link you can find the website where you can test your queries on the NEEM: https://data.open-ease.org/2021-robot-mooc-ex2 The link sends you to an real query interface for a recorded episode. With the *Examples* button you can try different queries on the episode. ### Exercise - Episode Structure Write the following queries: 1. Write a query that yields the shortest action in which a transporting task was executed. Tip: You can use ```event_interval(Action, Start, End)```, where *Action* is the given action, and *Start* and *End* the start- and endtimepoint of the action. Use ```number(End)``` to ensure that *Start* and *End* is bound to a rational number. To obtain the executed task you can use the property *dul:executesTask*. The type of the task is *soma:'Transporting'*. 2. Write a query that yields the type of tasks that are executed in actions in which an agent manipulates an item. Tip: Being an item in an action is the role an object plays when a task is executed, the roles of a task are denoted by the *dul:isTaskOf* property. The type of the role of manipulated items is *soma:'Item'*, and, for agents in a task, the role type is *soma:'AgentRole'*. ### Solutions #### Exercise 1. ``` % Get all events and their durations findall([Duration, Evt], ( event_interval(Evt, Begin, End), % Ensure that the Action executes a Transporting task triple(Evt, dul:'executesTask', Tsk), triple(Tsk, rdf:type, soma:'Transporting'), % Ensure that the end is bound to a number number(End), % Calculate the duration Duration is End - Begin ), Durations), % Get the minimal member of the solutions min_member([MinDuration, ShortestEvent], Durations). ``` 2. ``` % Get the executed Task triple(Act, dul:'executesTask', Tsk), % Ensure that there is a agent % and item during the Action triple(Tsk, dul:isTaskOf, ObjectRole), triple(ObjectRole, rdf:type, soma:'Item'), triple(Tsk, dul:isTaskOf, AgentRole), triple(AgentRole, rdf:type, soma:'AgentRole'), % Return the type of the task triple(Tsk, rdf:type, TskType). ```