Webapps packaging and files layout.

[Romain Beauxis]

Hi guys,

I have tried today the wordpress package.

I am startled by the fact that a package with such a
huge popcon provides a directory in /usr/share for
puting the themes and plugins (see #311821).

Additionally, themes and plugins upload from the admin
interface simply do not work out of the box...

This kind of issue is due to the maintainer not understanding
how the layout of the files should be. As explained in the
webapp policy, the files of the web application should be split
and placed in several folders, according to their purpose:
* arch independent files
* arch dependent files
* data files
* customizable files
* user-uploaded content

Now, the usual problem is that webapp softwares are not
generaly designed with this requirement in mind. Hence, it
is not clear for the packager how to split the files in several
locations and still get a functioning webapp.

For this reason, we end with issue like the one mentioned in
wordpress. At the end of the report mentioned above, I
explained the solution I have been using for mediawiki, spip
and roundcube. It simply concists in mirroring the entire file
layout at each different location.

For instance, in mediawiki, we have for /usr/share/mediawiki:
9:07 ***@leonard /# ll /usr/share/mediawiki
total 78K
lrwxrwxrwx 1 root root 36 1 mai 12:54 AdminSettings.php -> /var/lib/mediawiki/AdminSettings.php
-rw-r--r-- 1 root root 3,3K 5 mai 2009 api.php
lrwxrwxrwx 1 root root 25 1 mai 12:54 config -> /var/lib/mediawiki/config/
lrwxrwxrwx 1 root root 29 1 mai 12:54 extensions -> /var/lib/mediawiki/extensions/
lrwxrwxrwx 1 root root 25 1 mai 12:54 images -> /var/lib/mediawiki/images/
-rw-r--r-- 1 root root 3,2K 21 mars 2009 img_auth.php
drwxr-xr-x 13 root root 4,9K 1 mai 12:54 includes/
-rw-r--r-- 1 root root 4,2K 20 mars 2009 index.php
-rw-r--r-- 1 root root 5,8K 8 mars 11:42 install-utils.inc
drwxr-xr-x 4 root root 232 1 mai 12:54 languages/
lrwxrwxrwx 1 root root 36 1 mai 12:54 LocalSettings.php -> /var/lib/mediawiki/LocalSettings.php
drwxr-xr-x 12 root root 5,5K 1 mai 12:54 maintenance/
-rw-r--r-- 1 root root 3,0K 21 mars 2009 opensearch_desc.php
-rw-r--r-- 1 root root 9,0K 9 déc. 2008 profileinfo.php
-rw-r--r-- 1 root root 383 21 mars 2009 redirect.php
-rw-r--r-- 1 root root 91 31 janv. 2005 redirect.phtml
drwxr-xr-x 9 root root 672 1 mai 12:54 skins/
-rw-r--r-- 1 root root 603 7 janv. 2009 StartProfiler.php
-rw-r--r-- 1 root root 4,8K 8 mars 16:49 thumb.php
-rw-r--r-- 1 root root 1,4K 5 nov. 2008 trackback.php
-rw-r--r-- 1 root root 88 31 janv. 2005 wiki.phtml

and in /var/lib/mediawiki:
9:07 ***@leonard /# ll /var/lib/mediawiki
total 0
lrwxrwxrwx 1 root root 32 1 mai 12:54 AdminSettings.php -> /etc/mediawiki/AdminSettings.php
lrwxrwxrwx 1 root root 28 1 mai 12:54 api.php -> /usr/share/mediawiki/api.php
drwx------ 2 www-data www-data 112 1 mai 12:54 config/
drwxr-xr-x 2 root root 352 1 mai 12:54 extensions/
drwx------ 2 www-data www-data 48 16 avril 15:00 images/
lrwxrwxrwx 1 root root 33 1 mai 12:54 img_auth.php -> /usr/share/mediawiki/img_auth.php
lrwxrwxrwx 1 root root 29 1 mai 12:54 includes -> /usr/share/mediawiki/includes/
lrwxrwxrwx 1 root root 30 1 mai 12:54 index.php -> /usr/share/mediawiki/index.php
lrwxrwxrwx 1 root root 38 1 mai 12:54 install-utils.inc -> /usr/share/mediawiki/install-utils.inc
lrwxrwxrwx 1 root root 30 1 mai 12:54 languages -> /usr/share/mediawiki/languages/
lrwxrwxrwx 1 root root 32 1 mai 12:54 LocalSettings.php -> /etc/mediawiki/LocalSettings.php
lrwxrwxrwx 1 root root 32 1 mai 12:54 maintenance -> /usr/share/mediawiki/maintenance/
lrwxrwxrwx 1 root root 40 1 mai 12:54 opensearch_desc.php -> /usr/share/mediawiki/opensearch_desc.php
lrwxrwxrwx 1 root root 36 1 mai 12:54 profileinfo.php -> /usr/share/mediawiki/profileinfo.php
lrwxrwxrwx 1 root root 33 1 mai 12:54 redirect.php -> /usr/share/mediawiki/redirect.php
lrwxrwxrwx 1 root root 35 1 mai 12:54 redirect.phtml -> /usr/share/mediawiki/redirect.phtml
lrwxrwxrwx 1 root root 26 1 mai 12:54 skins -> /usr/share/mediawiki/skins/
lrwxrwxrwx 1 root root 38 1 mai 12:54 StartProfiler.php -> /usr/share/mediawiki/StartProfiler.php
lrwxrwxrwx 1 root root 30 1 mai 12:54 thumb.php -> /usr/share/mediawiki/thumb.php
lrwxrwxrwx 1 root root 34 1 mai 12:54 trackback.php -> /usr/share/mediawiki/trackback.php
lrwxrwxrwx 1 root root 31 1 mai 12:54 wiki.phtml -> /usr/share/mediawiki/wiki.phtml

(Yes, skins/ should be there I realized that..)

The reason for the mirroring is that usual php and other
language's include are relative: if a file from
/var/lib/mediawiki includes a files that is a link to
/usr/share/mediawiki, then the relative
inclusion path moves there.

In order to make sure the relative inclusion works back
and forth, you mirror everything at each place..

As I read it, the current policy tells you to split the files
but does not tell you how to reassemble them in order
to have a functional structure. I would like to add this
solution to the policy in order to eventually get rid of the
issues like the one in wordpress.

At some point I also considered writing a debhelper to
simplify this; you would give it the list of files and location
and it would build the different mirrors according to this.

What do you think ?

Romain

[Romain Beauxis]

Hi guys,

I have tried today the wordpress package.

I am startled by the fact that a package with such a
huge popcon provides a directory in /usr/share for
puting the themes and plugins (see #311821).

Additionally, themes and plugins upload from the admin
interface simply do not work out of the box...

This kind of issue is due to the maintainer not understanding
how the layout of the files should be. As explained in the
webapp policy, the files of the web application should be split
and placed in several folders, according to their purpose:
* arch independent files
* arch dependent files
* data files
* customizable files
* user-uploaded content

Now, the usual problem is that webapp softwares are not
generaly designed with this requirement in mind. Hence, it
is not clear for the packager how to split the files in several
locations and still get a functioning webapp.

For this reason, we end with issue like the one mentioned in
wordpress. At the end of the report mentioned above, I
explained the solution I have been using for mediawiki, spip
and roundcube. It simply concists in mirroring the entire file
layout at each different location.

For instance, in mediawiki, we have for /usr/share/mediawiki:
9:07 ***@leonard /# ll /usr/share/mediawiki
total 78K
lrwxrwxrwx 1 root root 36 1 mai 12:54 AdminSettings.php -> /var/lib/mediawiki/AdminSettings.php
-rw-r--r-- 1 root root 3,3K 5 mai 2009 api.php
lrwxrwxrwx 1 root root 25 1 mai 12:54 config -> /var/lib/mediawiki/config/
lrwxrwxrwx 1 root root 29 1 mai 12:54 extensions -> /var/lib/mediawiki/extensions/
lrwxrwxrwx 1 root root 25 1 mai 12:54 images -> /var/lib/mediawiki/images/
-rw-r--r-- 1 root root 3,2K 21 mars 2009 img_auth.php
drwxr-xr-x 13 root root 4,9K 1 mai 12:54 includes/
-rw-r--r-- 1 root root 4,2K 20 mars 2009 index.php
-rw-r--r-- 1 root root 5,8K 8 mars 11:42 install-utils.inc
drwxr-xr-x 4 root root 232 1 mai 12:54 languages/
lrwxrwxrwx 1 root root 36 1 mai 12:54 LocalSettings.php -> /var/lib/mediawiki/LocalSettings.php
drwxr-xr-x 12 root root 5,5K 1 mai 12:54 maintenance/
-rw-r--r-- 1 root root 3,0K 21 mars 2009 opensearch_desc.php
-rw-r--r-- 1 root root 9,0K 9 déc. 2008 profileinfo.php
-rw-r--r-- 1 root root 383 21 mars 2009 redirect.php
-rw-r--r-- 1 root root 91 31 janv. 2005 redirect.phtml
drwxr-xr-x 9 root root 672 1 mai 12:54 skins/
-rw-r--r-- 1 root root 603 7 janv. 2009 StartProfiler.php
-rw-r--r-- 1 root root 4,8K 8 mars 16:49 thumb.php
-rw-r--r-- 1 root root 1,4K 5 nov. 2008 trackback.php
-rw-r--r-- 1 root root 88 31 janv. 2005 wiki.phtml

and in /var/lib/mediawiki:
9:07 ***@leonard /# ll /var/lib/mediawiki
total 0
lrwxrwxrwx 1 root root 32 1 mai 12:54 AdminSettings.php -> /etc/mediawiki/AdminSettings.php
lrwxrwxrwx 1 root root 28 1 mai 12:54 api.php -> /usr/share/mediawiki/api.php
drwx------ 2 www-data www-data 112 1 mai 12:54 config/
drwxr-xr-x 2 root root 352 1 mai 12:54 extensions/
drwx------ 2 www-data www-data 48 16 avril 15:00 images/
lrwxrwxrwx 1 root root 33 1 mai 12:54 img_auth.php -> /usr/share/mediawiki/img_auth.php
lrwxrwxrwx 1 root root 29 1 mai 12:54 includes -> /usr/share/mediawiki/includes/
lrwxrwxrwx 1 root root 30 1 mai 12:54 index.php -> /usr/share/mediawiki/index.php
lrwxrwxrwx 1 root root 38 1 mai 12:54 install-utils.inc -> /usr/share/mediawiki/install-utils.inc
lrwxrwxrwx 1 root root 30 1 mai 12:54 languages -> /usr/share/mediawiki/languages/
lrwxrwxrwx 1 root root 32 1 mai 12:54 LocalSettings.php -> /etc/mediawiki/LocalSettings.php
lrwxrwxrwx 1 root root 32 1 mai 12:54 maintenance -> /usr/share/mediawiki/maintenance/
lrwxrwxrwx 1 root root 40 1 mai 12:54 opensearch_desc.php -> /usr/share/mediawiki/opensearch_desc.php
lrwxrwxrwx 1 root root 36 1 mai 12:54 profileinfo.php -> /usr/share/mediawiki/profileinfo.php
lrwxrwxrwx 1 root root 33 1 mai 12:54 redirect.php -> /usr/share/mediawiki/redirect.php
lrwxrwxrwx 1 root root 35 1 mai 12:54 redirect.phtml -> /usr/share/mediawiki/redirect.phtml
lrwxrwxrwx 1 root root 26 1 mai 12:54 skins -> /usr/share/mediawiki/skins/
lrwxrwxrwx 1 root root 38 1 mai 12:54 StartProfiler.php -> /usr/share/mediawiki/StartProfiler.php
lrwxrwxrwx 1 root root 30 1 mai 12:54 thumb.php -> /usr/share/mediawiki/thumb.php
lrwxrwxrwx 1 root root 34 1 mai 12:54 trackback.php -> /usr/share/mediawiki/trackback.php
lrwxrwxrwx 1 root root 31 1 mai 12:54 wiki.phtml -> /usr/share/mediawiki/wiki.phtml

(Yes, skins/ should be there I realized that..)

The reason for the mirroring is that usual php and other
language's include are relative: if a file from
/var/lib/mediawiki includes a files that is a link to
/usr/share/mediawiki, then the relative
inclusion path moves there.

In order to make sure the relative inclusion works back
and forth, you mirror everything at each place..

As I read it, the current policy tells you to split the files
but does not tell you how to reassemble them in order
to have a functional structure. I would like to add this
solution to the policy in order to eventually get rid of the
issues like the one in wordpress.

At some point I also considered writing a debhelper to
simplify this; you would give it the list of files and location
and it would build the different mirrors according to this.

What do you think ?

Romain