Since the release of Libreoffice 4.2, the new flat and monochrome icon theme Sifr was introduced, which has received a lot of positive feedback from the community for its modern design. So in order to test the icon set’s perfection, we have run an icon test similar to two previous tests done in 2013 with Tango and Oxygen. Here are the results from the test.
Method
The icon test on our User Weave test platform aims to analyze icons using quantitative data instead of subjective preferences. All icons are presented in random order, with every term tested one at a time. The participant’s task is to find the appropriate icon to the presented term as quickly as possible. Learn more about this method in Extracting the DNA of icons.
Figure 1: Libreoffice with Sifr icons.
The current test consisted of 24 icons from LibreOffice’s standard toolbar.
We used the tooltip labels to describe the function. The text was localized in English (n=240 participants), German (n=72), French (n=38), Polish (n=11), Italian (n=8), and Hebrew (no one chose this language).
Results
Icon test
Figure 2 shows the calculated quality indicator for the Sifr icons (red) versus the (reanalyzed) sets with Tango and Oxygen icons (gray and black). The lower the indicator the less mistaken the icon was and the quicker it was chosen
Figure 2: Calculated quality indicator.
Typically, some icons are mutually mixed-up. The next image shows the percentage of those false associations. On the horizontal x-axis, the presented term is shown with the chosen image on the vertical y-axis. Gray shades are logarithmic scale.
Figure 3: Mix-ups: x-axis shows the term, y-axis the percentage of false assigned images. Gray shades are log scaled.
If you want to rerun our analysis we provide all raw data and R scripts for the analysis of the icon test (run evaluate.r): Sifr_Icontest.tar.gz
Descriptives
After the icon test we asked the participants a few demographic questions.
After the icon selection test was completed, we asked the participants a few demographic questions.
Since the test is about icons, it might be interesting to know which icon set users select deliberately: “What icon set do you use in Libreoffice?” The answers comprised of the distributed sets for all operating systems as well as the option to answer with “The predefined default icon set”. The following table combines these result with the question “How old are you?” in six categories. The last row and column contain the total percentage for age or icon set.
Table 1: Use of icon sets by age.| Icon set | 20- 30 | 30- 40 | 40- 50 | 50- 60 | >60 | Total [%] | |
|---|---|---|---|---|---|---|---|
| Crystal | 0.00 | 0.66 | 0.99 | 0.33 | 0.66 | 0.00 | 2.65 |
| Galaxy | 0.00 | 0.99 | 2.65 | 0.66 | 1.32 | 0.66 | 6.29 |
| Hicontrast | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.33 | 0.33 |
| Human | 0.33 | 0.66 | 0.99 | 0.00 | 0.33 | 0.00 | 2.32 |
| Oxygen | 0.99 | 0.99 | 2.98 | 1.66 | 0.66 | 0.33 | 7.62 |
| Predefined | 0.66 | 13.25 | 16.89 | 7.62 | 4.97 | 2.32 | 45.70 |
| Sifr | 1.32 | 7.62 | 7.62 | 3.64 | 0.33 | 0.00 | 20.53 |
| Tango | 0.00 | 4.64 | 3.31 | 2.65 | 0.99 | 0.66 | 12.25 |
| Total [%] | 3.31 | 29.14 | 35.76 | 16.56 | 9.27 | 4.30 | n=302 |
We then asked “Which is your primary operating system?”. Again, the table combines this data with the chosen icon set.
Table 2: Use of icon sets by operating systems.| Icon set | Linux (GTK+) | Linux (Qt) | Linux (other) | Mac OS | Win | Total [%] |
|---|---|---|---|---|---|---|
| Crystal | 0.00 | 0.66 | 0.00 | 0.00 | 1.99 | 2.65 |
| Galaxy | 0.99 | 0.99 | 0.33 | 0.00 | 3.97 | 6.29 |
| Hicontrast | 0.00 | 0.00 | 0.00 | 0.00 | 0.33 | 0.33 |
| Human | 1.66 | 0.00 | 0.66 | 0.00 | 0.00 | 2.32 |
| Oxygen | 0.33 | 5.30 | 0.66 | 0.00 | 1.32 | 7.62 |
| Predefined | 10.26 | 6.29 | 3.64 | 2.98 | 22.85 | 46.36 |
| Sifr | 6.29 | 4.30 | 1.66 | 2.65 | 5.96 | 21.19 |
| Tango | 5.96 | 0.66 | 0.99 | 0.66 | 3.97 | 12.25 |
| Total [%] | 25.50 | 18.21 | 7.95 | 6.29 | 41.06 | n=302 |
And finally we wanted to know “How often do you use the following LibreOffice Application?”.
Table 3: Average usage of applications from 1=never to 5=daily.| Writer | Calc | Impress | Draw | Base | Math |
|---|---|---|---|---|---|
| 4.00 | 3.40 | 2.31 | 1.97 | 1.35 | 1.53 |
Unfortunately it is not possible to link any of the demographic results to the data from the icon test.
If you want to rerun our analysis we provide all raw data and R scripts for descriptives (run descriptives.r): Sifr_Descriptives.tar.gz
Discussion
Sifr is a popular icon set. It is chosen deliberately by about 20% of the participants, not only by younger users and not only by Linux or MacOS users. And it performs pretty well. The results are similar to Oxygen and Tango which are known to be great icon sets. We treat this as another confirmation for out method.
On the other hand, this particular selection of icons from the Libreoffice standard toolbar shows some issues. There is a significant mix-up between the functions for spell checking, and Copy/Paste are confused more often than in the other icon sets. This is a common problem of monochrome designed icons that lack on the color dimension. Rather related to Libreoffice the functions Gallery, Navigator, Data Sources, and Page Preview are barely understood.
We documented the issues last year:
- About Antiquated Metaphors in Icons
- More is worse: About Detail in Icons
- What is a pencil used for?
- Can a direction in time be displayed by spatial signs?
- Where does the Navigator lead you?
With the conclusion and recommendations:
- Semiotics in Usability: Guidelines for the Development of Icon Metaphors
- Quick wins: Conclusions of the Libreoffice icon test
- Tip the tool: How to label toolbar functions
Next steps
The Libreoffice design team is working hard on toolbars to improve the UX for the upcoming release 4.4. The toolbar is decluttered in that less frequently used functions are removed and more relevant ones have been added. Tooltips are being standardized and updated. About Sifr, we have decided to make it the default for Mac OS in the upcoming release.
