The ERL Emergency Service Robots is composed of one major tournament, held in Piombino 15-23 September 2017. Teams participate and get scores based on their performances. A final score is computed for each team, per Task Benchmark (TBM) and Functionality Benchmark (FBM), using the best run of the trial. Teams are ranked based on their final score.
More information can be found on ERL Emergency Rulebook.
Prizes for the top teams per TBM were awarded during the Awards Ceremony held at Giardini Pro Patria, in Piombino, Italy; and prizes for the best teams per FBM were awarded during the European Robotics Forum (ERF) in Tampere, 13-15 March 2018.
The final scoring and standings for the ERL Emergency Service Robots Challenge are the ones below:
Task Benchmarks
TBM1: The Grand Challenge (Land +Sea + Air)
|
Sorted Based on best run |
||
|
Team |
A |
P |
|
Telerob + Universitat de Girona + ISEP/INSESC TEC |
67 |
0 |
|
Robdos-IMM + IIS Piombino CVP |
46 |
0 |
|
Raptors + Oubot |
40 |
0 |
|
Tuscany Robotics |
38 |
3 |
|
Bebot + AUV TomKyle + HSR Search and Rescue |
33 |
2 |
A - Achievements
P – Penalties
Maximum achievements: 98
ERL Emergency 2017 - TBM1-results
TBM2: Survey the building and search for missing workers (Land +Air)
|
Sorted Based on best run |
||
|
Team |
A |
P |
|
IMM + IIS Piombino CVP |
23 |
0 |
|
Raptors |
21 |
0 |
|
Telerob + ISEP/INSESC TEC |
19 |
0 |
|
ENSTA Team |
16 |
1 |
|
ENSTA Bretagne |
13 |
1 |
|
Bebot + HSR Search and Rescue |
8 |
2 |
|
ETH Zürich ASL-RSL |
4 |
2 |
Maximum achievements: 40
ERL Emergency 2017 - TBM2 -results
TBM3: Pipe inspection and search for missing workers (Sea +Air)
|
Sorted Based on best run |
||
|
Team |
A |
P |
|
Universitat de Girona + ISEP/INSESC TEC |
24 |
0 |
|
Tuscany Robotics |
16 |
0 |
|
AUV TomKyle + HSR Search and Rescue |
16 |
1 |
|
Robdos + IIS Piombino CVP |
14 |
0 |
|
Raptors + Oubot |
9 |
0 |
|
ENSTA Bretagne |
6 |
0 |
|
ENSTA Team |
0 |
0 |
Maximum achievements: 64
ERL Emergency 2017 - TBM3- results
TBM4: Stem the leak (Land +Sea)
|
Sorted Based on best run |
||
|
Team |
A |
P |
|
Telerob + Universitat de Girona |
24 |
0 |
|
Raptors + Oubot |
17 |
0 |
|
Bebot + AUV TomKyle |
11 |
3 |
|
Robdos-IMM |
10 |
0 |
|
ENSTA Team |
7 |
2 |
|
ENSTA Bretagne |
6 |
1 |
Maximum achievements: 58
ERL Emergency 2017 - TBM4 - results
Functionality Benchmarks
FBM1: Mapping Functionality (Land + Air)
| Team | RMSE | Ground coverage | Score |
| IMM (Land)+ IIS Piombino CVP (Air) | 1 | 0.50 | 0.75 |
| ENSTA Bretagne (Land+Air) | 1 | 0 | 0.50 |
| Telerob (Land)+ISEP/INSESC TEC (Air) | 0.91333 | 0 | 0.46 |
| ETH (Land+Air) | 0.82667 | 0 | 0.42 |
| Bebot (Land) + HSR Search and Rescue (Air) | 0 | 0 | 0 |
| ENSTA Team(Land+Air) | 0 | 0 | 0 |
| Raptors (Land+Air) | 0 | 0 | 0 |
Maximum Score: 1
SCORE=(RMSE+ Ground coverage)/2
ERL Emergency 2017 - FBM1 -results
FBM2: Object Recognition (Land + Air)
| Sorted based on best run | ||||
| Team | OPIs found | Precision | Recall | Index |
| Telerob (Land)+ISEP/INSESC TEC (Air) | 9 | 1 | 0.81818 | 0.90000 |
| Raptors (Land+Air) | 8 | 1 | 0.72727 | 0.84211 |
| IMM (Land)+ IIS Piombino CVP (Air) | 7 | 1 | 0.63636 | 0.77778 |
| ENSTA Team(Land+Air) | 5 | 1 | 0.45455 | 0.62500 |
| ENSTA Bretagne (Land+Air) | 4 | 1 | 0.36364 | 0.53333 |
| Bebot (Land) + HSR Search and Rescue (Air) | 1 | 1 | 0.09091 | 0.16667 |
| ETH (Land+Air) | 0 | 0 | 0 | 0 |
Maximum Index: 1
Index= F-mesure = 2*(Precission/Recall)/(Precision+Recall)
for Beta=1
(See Rulebook for equations).
ERL Emergency 2017 - FBM2 - results
FBM3: Object Detection (Sea)
| Sorted based on best run | ||||||||
| Team | Buoy 1 |
Buoy 2 |
Buoy 3 | Buoy 4 |
Buoy 5 |
CI | CC |
Score |
| Universitat de Girona | Yes | Yes | No | Yes | Yes | 0 | 4 | 4 |
| AUV Team Tomkyle | Yes | Yes | No | Yes | No | 0 | 3 | 3 |
| Tuscany Robotics Team | No | No | Yes | No | Yes | 0 | 2 | 2 |
| ENSTA Bretagne | Yes | No | No | No | No | 0 | 1 | 1 |
| ENSTA Team | No | No | No | No | No | 0 | 0 | 0 |
| OUBOT | No | No | No | No | No | 0 | 0 | 0 |
| Robdos | No | No | No | No | No | 0 | 0 | 0 |
CI - Correctly Identified buoys (via the black number)
CC – Correctly Classified buoys (without the detection of the black number)
SCORE=2.5*CI+CC
ERL Emergency 2017 - FBM3 - results
FBM4: Vertical Wall Mapping Functionality (Air)
| Team | RMSE | MaxErr | Index |
| IIS Piombino CVP | 0.31 | 18 | 0.98 |
| ISEP/INESC TEC | 0.45 | 18 | 0.97 |
| HSR Search and Rescue | 13.41 | 18 | 0.25 |
| Tuscany Robotics Team | - | - | 0 |
| RAPTORS | - | - | 0 |
Maximum Index: 1
ERL Emergency 2017 - FBM4 -results