Issue

This is coming up working on getting "gnome-software" working on Nix. AppStream is an XML data format to provide meta data listings on "apps" available to a distribution that is used by "gnome-software" to provide app listings in a distro agnostic way.

AppStream Metadata

To work with AppStream, a Linux distribution needs to provide an XML file that has a list of "components". The resulting XML looks like this:

​<?xml version="1.0"?>
​<components version="0.6">
​  <component type="application">
​    <id>firefox.desktop</id>
​    <pkgname>firefox-bin</pkgname>
​    <name>Firefox</name>
​    <name lang="en_GB">Firefoux</name>
​    <summary>Web browser</summary>
​    <summary lang="fr_FR">Navigateur web</summary>
​    <project_license>MPL-2</project_license>
​    <keywords>
​      <keyword>internet</keyword>
​      <keyword>web</keyword>
​      <keyword>browser</keyword>
​      <keyword lang="fr_FR">navigateur</keyword>
​    </keywords>
​    <icon type="stock">web-browser</icon>
​    <icon type="cached">firefox.png</icon>
​    <categories>
​      <category>network</category>
​      <category>web</category>
​    </categories>
​    <mimetypes>
​      <mimetype>text/html</mimetype>
​      <mimetype>text/xml</mimetype>
​      <mimetype>application/xhtml+xml</mimetype>
​      <mimetype>application/vnd.mozilla.xul+xml</mimetype>
​      <mimetype>text/mml</mimetype>
​      <mimetype>application/x-xpinstall</mimetype>
​      <mimetype>x-scheme-handler/http</mimetype>
​      <mimetype>x-scheme-handler/https</mimetype>
​    </mimetypes>
​    <url type="homepage">http://www.mozilla.com</url>
​    <screenshots>
​      <screenshot type="default">
​        <image type="source" width="800" height="600">http://www.awesomedistro.example.org/en_US/firefox.desktop/main.png</image>
​        <image type="thumbnail" width="200" height="150">http://www.awesomedistro.example.org/en_US/firefox.desktop/main-small.png</image>
​      </screenshot>
​    </screenshots>
​    <provides>
​      <binary>firefox</binary>
​    </provides>
​  </component>
​  <component>
​    <id>pulseaudio</id>
​    <name>PulseAudio</name>
​    <summary>The PulseAudio sound server</summary>
​    <url type="homepage">http://www.freedesktop.org/wiki/Software/PulseAudio/</url>
​    <project_license>GPLv2</project_license>
​    <provides>
​      <library>libpulse-simple.so.0</library>
​      <library>libpulse.so.0</library>
​      <binary>start-pulseaudio-kde</binary>
​      <binary>start-pulseaudio-x11</binary>
​    </provides>
​    <release version="2.0"/>
​  </component>
​  <component type="font">
​    <id>LinLibertine_M.otf</id>
​    <name>Libertine</name>
​    <summary>Linux Libertine Open fonts</summary>
​    <font_classifier>Mono</font_classifier>
​    <font_parent>Libertine</font_parent>
​  </component>
​  <!-- more components here! -->
​</components>

Some of the fields like "id", "packagename", "url", and "summary" we already have analogues in Nix. The issue going forward is going to be generating the rest of the data. Luckily most upstream apps already provide this data in the form of "appdata" XML files.

Upstream metadata

Most GNOME apps and some other apps already provide most of this data in the form of "appdata" XML. This is a short file that has some info on metadata like what libraries, fonts, binaries, etc it provides. The result looks like this:

​<?xml version="1.0" encoding="UTF-8"?>
​<component>
​  <id>foobar</id>
​  <name>Foo Bar</name>
​  <summary>A foo-ish bar</summary>
​  <url type="homepage">http://www.example.org</url>
​  <metadata_license>CC0-1.0</metadata_license>
​
​  <provides>
​    <library>libfoobar.so.2</library>
​    <font>foo.ttf</font>
​    <binary>foobar</binary>
​  </provides>
​  <releases>
​    <release version="1.2" date="2015-02-16" />
​  </releases>
​  <developer_name>FooBar Team</developer_name>
​</component>

In a normal distribution, each XML metadata file is installed either in "/usr/share/metainfo/" or "/usr/share/appdata/". With Nix, these files should already be installed in relevant derivations under /nix/store/.../share/appdata/ and /nix/store/.../share/metainfo. In addition, there are some other files that provide data that would be useful to keep track of:

• /share/appdata/
• /share/metainfo/
• /share/pixmaps/
• /share/icons/
• /share/${app_name}/icons/
• /share/applications/
• /share/applications/kde4/

Derivations that generate these folders need to be added to some index to keep track of AppStream "apps". This is going to be similar to how we were doing "command-not-found" support before issue NixOS/nixos-channel-scripts#4.

Proposal

So, my basic thinking right now is that Hydra would generate a list of all files within a derivation and upload it in the form of a *.narlist as in issue NixOS/nixos-channel-scripts#4. A NixOS channel script could then be run like "generate-programs-index.pl" that would generate the above AppStream metadata. Each channel would need to have its own AppStream list to ensure a correct index.

Each derivation that has any of the above file paths in it would cause a "generate-appstream-index.pl" script to be run with the derivation's out path. Each derivation would generate a "component" like above from the different folders output in the derivation. Each component would be added to the list of components in an "appstream.xml" file. That appstream.xml file could then be added to the "nixexprs.tar.xz" like "programs.sqlite" was previously. The generated appstream.xml would then be available at "/nix/var/nix/profiles/per-user/root/channels/nixos/appstream.xml".

The challenge would be making sure that "appstream-glib" is able to detect our appstream.xml file. Because it's not technically a derivation, this is basically going to require hard coding that path in like command-not-found does. This would require modifying the appstream-glib derivation to trick it into looking in /nix/var/nix/profiles/per-user/root/channels/nixos/.

Remaining Issues

There is other data that would be useful for appstream to have available. This includes the "/usr/share/applications/" and "/usr/share/icons/". Both of these are going to be much trickier because there's no easy way to put them in an XML file like with "appstream.xml" above. In addition, we don't want to put too much in "nixexprs.tar.xz" and things like icons are inevitably going to bloat the file size.

Conclusion

I am not completely familiar with how Hydra and NixOS are built so any suggestions or corrections are welcome for this proposal. It requires some extra steps in the build process but I think it should be doable.