Discussion:
Getting a package added to the debian package repository: WACS
Beaky, The WACS Man
2010-02-18 08:49:26 UTC
Permalink
Greetings Folks

I hope this is the right mailing list to contact - it was recommended to
me as the best point of contact by some debian developers I met at the
recent Linux.Conf.Au in Wellington NZ.

I'm the lead developer on a free software package called WACS (Web-based
Adult Content Server) which is in essence a complete Content Management
System targeted at Adult content. It is designed to be used both for
private collections and as a basis for running commercial web sites.

It includes:

- a Complete search/retrieval system
- Extensive web-based content/collection management tools
- Extensive APIs implemented in both Perl and PHP5
- a customisable example web site template
- command line collection management tools
- a download system
- Complete database schema with documentation
- Massive Documentation library (approx 450 pages)

The project has been under development since 2004, on sourceforge since
2005, and has a live free demonstration site at www.pinkmetallic.com.
More information on WACS can be found at http://wacsip.sourceforge.net
and at https://launchpad.net/wacs/

We're just approaching a new release (0.8.5) which will hopefully be
ready to go around the middle of March. Since we completed the
web-based installer for the previous release (0.8.4), we now feel that
the package has the maturity to be offered to Linux distributions for
consideration for inclusion in their repositories.

As far as we know, we are the only major Open Source project out there
concentrating on making a free Web CMS suitable for adult content. We
definitely believe our system is extremely capable and compares well to
the commercial alternatives.

It is licensed under the GPLv3 (apart from a few included javascripts
etc which are GPLv2). The documentation is also available as DocBook
XML source and is licensed under the Creative Commons By Attribution
license.

We've already done packaging work ourselves for both Fedora (RPM) and
Ubuntu (DEB), but I understand that Debian prefer to have a packager
other than the developers themselves do this work (and I imagine other
distributions feel the same when moving software into their
repositories).

I would very much like to open a dialog with the debian webapps team
about whether WACS is now, as I believe, suitable for distribution on a
much wider scale. I would also like to have discussions with a
prospective packager ahead of the 0.8.5 release with a hope of getting
any changes they would wish to make to the configuration files into the
official release distribution.

Thank you for reading. I look forward to hearing from you.

Cheers
Beaky.


--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@kennedy.bevteccom.co.uk
sean finney
2010-02-18 18:35:41 UTC
Permalink
hi there,

fwiw i'm going to side-step any discussion related to appropriateness
of content and would appreciate others doing the same (or at least take
it to another list). but at the same time, please no more links
to "example" sites!

On Thu, Feb 18, 2010 at 08:49:26AM +0000, Beaky, The WACS Man wrote:
> We've already done packaging work ourselves for both Fedora (RPM) and
> Ubuntu (DEB), but I understand that Debian prefer to have a packager
> other than the developers themselves do this work (and I imagine other
> distributions feel the same when moving software into their
> repositories).

i don't think there's any rule against the developers doing the packaging
work, though you will need someone to sponsor your package at least
initially for it to be accepted into the archive.

> I would very much like to open a dialog with the debian webapps team
> about whether WACS is now, as I believe, suitable for distribution on a
> much wider scale. I would also like to have discussions with a
> prospective packager ahead of the 0.8.5 release with a hope of getting
> any changes they would wish to make to the configuration files into the
> official release distribution.

i think "team" is a bit of a misnomer here, as this list is more of a
discussion area for the common set of problems that one has with web
application packaging. that said, you might still find someone here
interested in sponsoring or co-maintaining your package (i'm not
volunteering though). failing that, you could also try the debian-mentors
list.


sean
Beaky, The WACS Man
2010-02-18 23:24:40 UTC
Permalink
Sean

On Thu, 2010-02-18 at 19:35 +0100, sean finney wrote:
> fwiw i'm going to side-step any discussion related to appropriateness
> of content and would appreciate others doing the same (or at least take
> it to another list). but at the same time, please no more links
> to "example" sites!

Sorry, I am not intending any upset.

The example site is completely genuine, only contains material licensed
under the creative commons by attribution 3.0 license and is actually
running the WACS software and basically nothing else. It is not some
devious front for a commercial organisation I assure you. It is 100%
genuine and true to the ideals of free software.

> i don't think there's any rule against the developers doing the packaging
> work, though you will need someone to sponsor your package at least
> initially for it to be accepted into the archive.

OK, then it's obviously not quite as I was led to believe, but that's
fine, just let me know who I should be talking to. I do however feel
that due to the nature of the subject having a second person go over it
will be distinctly advantageous in this case.

For instance, as things stand, I protect almost all files with mode 660
to the group of package administrators which causes lintian to get very
upset. I think that is the right thing to do and that the icons,
thumbnails and documentation should not be available for all users on
the system to read. That obviously runs counter to the normal
permissions mentality reflected in the lintian process.

> i think "team" is a bit of a misnomer here, as this list is more of a
> discussion area for the common set of problems that one has with web
> application packaging. that said, you might still find someone here
> interested in sponsoring or co-maintaining your package (i'm not
> volunteering though). failing that, you could also try the debian-mentors
> list.

OK, I'll hope that someone will choose to get back to me over the next
few days with some interest.

Given it's one of the biggest markets on the internet, I'm surprised
that no one has addressed this arena with free software previously (at
least not to my knowledge). I know a great many people sweep their
activities in this area under the carpet, but I really hope someone will
have the courage to come forward and help.

Right now this is a very important part of the internet as a whole where
the free software community is leaving users and developers solely in
the hands of (very predatory) proprietary software companies. I believe
that is wrong and we need, even if some find it distasteful, to make an
effort to support those users needs within free software. That is why I
and others have been working on this software, tuning and improving it,
and now seeking distribution for it so that we can say "here is another
important vertical market where free software has a viable alternative".

Not only that, but I believe this package is good enough to be better
than the commercial competition. With over 20 years experience as a
Unix/Linux system programmer and Oracle DBA, I believe (and the others
who have contributed) have given WACS has a good and solid design.

Cheers
Beaky



--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@kennedy.bevteccom.co.uk
Andrew McMillan
2010-02-19 00:59:07 UTC
Permalink
On Thu, 2010-02-18 at 23:24 +0000, Beaky, The WACS Man wrote:

> For instance, as things stand, I protect almost all files with mode 660
> to the group of package administrators which causes lintian to get very
> upset. I think that is the right thing to do and that the icons,
> thumbnails and documentation should not be available for all users on
> the system to read. That obviously runs counter to the normal
> permissions mentality reflected in the lintian process.

We can certainly talk about that sort of thing :-)

Why would you want to restrict local users from reading files which are
freely downloadable?


> > i think "team" is a bit of a misnomer here, as this list is more of a
> > discussion area for the common set of problems that one has with web
> > application packaging. that said, you might still find someone here
> > interested in sponsoring or co-maintaining your package (i'm not
> > volunteering though). failing that, you could also try the debian-mentors
> > list.
>
> OK, I'll hope that someone will choose to get back to me over the next
> few days with some interest.

I think that debian-mentors is probably a better place to start, other
than for discussion related to packaging a web application specifically.
While it's desirable to have up/downstream separation there's no real
problem with not having it, especially if you already have the package
in Ubuntu.

Sure, it's nice to have a packager who is separate from the developer,
but it's plenty common enough to have the roles combined, too.


> Given it's one of the biggest markets on the internet, I'm surprised
> that no one has addressed this arena with free software previously (at
> least not to my knowledge). I know a great many people sweep their
> activities in this area under the carpet, but I really hope someone will
> have the courage to come forward and help.

I'm sure only the people involved in the industry could say, but I doubt
that it fits so straightforwardly into the 'making the world a better
place' ethos of many free software developers.


> Right now this is a very important part of the internet as a whole where
> the free software community is leaving users and developers solely in
> the hands of (very predatory) proprietary software companies. I believe
> that is wrong and we need, even if some find it distasteful, to make an
> effort to support those users needs within free software. That is why I
> and others have been working on this software, tuning and improving it,
> and now seeking distribution for it so that we can say "here is another
> important vertical market where free software has a viable alternative".
>
> Not only that, but I believe this package is good enough to be better
> than the commercial competition. With over 20 years experience as a
> Unix/Linux system programmer and Oracle DBA, I believe (and the others
> who have contributed) have given WACS has a good and solid design.

That's great :-)

Is it usable for other things as well? I mean I don't really have the
content to load up into my local porn installation, but if it works
better than (say) Gallery I might be tempted to use it to display my
personal (and decidedly non-racy) photographs to the world...

You're marketing it as a Web 'Adult' CMS, but what is it about the
content that makes it special-cased for that particular vertical?

I doubt if anyone has a problem with this appearing in the archive (no
discrimination against fields of endeavour, right?), but what you will
not be able to do, is to make us feel guilty about looking sideways at
this package. I'm sure we are all already doing plenty for free
software in the way that suits us best, and very likely we're all doing
it voluntarily and are taking on more than we really have time for.

Your best approach here will be to seek to resolve technical issues and
questions and to leave the rhetoric about why this is a wonderful
package and how it will change the world to languish in the description
field of the debian/control file.

Cheers,
Andrew.

------------------------------------------------------------------------
andrew (AT) morphoss (DOT) com +64(272)DEBIAN
All the troubles you have will pass away very quickly.
------------------------------------------------------------------------
Beaky, The WACS Man
2010-02-19 09:13:16 UTC
Permalink
Andrew

On Fri, 2010-02-19 at 13:59 +1300, Andrew McMillan wrote:
> Why would you want to restrict local users from reading files which are
> freely downloadable?

Generally the act of downloading is a significant barrier to entry for
many people, particularly those who are not normally the "administrator"
of the system.

I was somewhat surprised to discover that a couple of the people
contributing to and using the project were "family men" who were using a
shared "family" computer for the installation. Since they had choosen
to do that, I felt the best that I could do in response to that type of
usage was to lock down the icons, documentation, and so on such that
another user on the system could not easily access them. Simply making
the web server a member of the wacs group ensured that it could read the
files as needed in order to work without them being open to casual
perusal. The same protections are actively applied to the content each
and every time it is manipulated through the collection management
tools. Anyone in the appropriate group (wacs) can of course read them.

Since the system, by default, requires authentication before showing
anything more than a very cryptic "you don't have permission" screen,
this does make it somewhat protected against casual curiosity.

> I think that debian-mentors is probably a better place to start, other
> than for discussion related to packaging a web application specifically.

I'll send a message to that list shortly.

> I'm sure only the people involved in the industry could say, but I doubt
> that it fits so straightforwardly into the 'making the world a better
> place' ethos of many free software developers.

That's certainly my motivation in releasing it.

> Is it usable for other things as well? I mean I don't really have the
> content to load up into my local porn installation, but if it works
> better than (say) Gallery I might be tempted to use it to display my
> personal (and decidedly non-racy) photographs to the world...

I use gallery2 myself for my own (non-racy) photos.

WACS simply doesn't handle single photos; it handles photostories - a
series of photos that are handled together as a slideshow and have a
determined order of display. It also handles video clips and includes
the infrastructure to index appearances on DVDs as well.

I can certainly see the potential for some other uses - somebody who had
scanned each and every image in a comic book collection for instance -
would probably find many of it's architectural components of significant
use. Similarly people collecting stills of a musician in concert might
wish to represent each concert as a unique photostory with a determined
order of display.

I'd love to see people develop those kind of projects using the
infra-structure we have provided. As someone with a passion for
photographing female nudity, that's not what "lights my candle" but I'm
in no way hostile to it.

> You're marketing it as a Web 'Adult' CMS, but what is it about the
> content that makes it special-cased for that particular vertical?

Basically the database schema. This is designed to represent the
relationships between models, sets, distributors and photographers.

It provides significant features related to keywording and attribute
mark-up - including a fairly unique retrospective keywording engine.
These features are all designed to be very configurable and would be
portable to many other applications, but of course the initial database
load provided with the distribution is of necessity very specific to the
target application.

We did actually discuss this technique and it's applicability to other
uses in some depth during the Photo Management BOF at linux.conf.au.

> is to make us feel guilty about looking sideways at this package.

Not really my intention. Just trying to explain that our motivations
for working on the package are very much the same as for the rest of the
free software movement. I've found over the years that merely because
of the subject matter people seem to become suspicious of ulterior
motives when there really aren't any.

> Your best approach here will be to seek to resolve technical issues and
> questions

I do see the permissions issue as one of the biggest. It's also what
lintian has the biggest problem with, and therefore the (technical) one
that is most likely to need consideration along the road to (hopefully)
acceptance.

I'm really interested in other people's views on whether protecting the
system components in this way is a reasonable precaution to take or an
unacceptable restriction on the other user's rights. I can see both
arguments.

Additionally I would very much hope that if the considered opinion is
that the protection is the right thing to do, that someone would
consider installing the package and doing an audit of what has been
installed and point out to us where there may be chinks in the armour.

I'm thinking about protecting the /usr/share/wacs/docs/ directory with
a .htaccess file that requires a username and password in the upcoming
0.8.5 release. That would partially remove one potential path to
circumvention of the group permissions protection, but it would be hard
to tie that into all of the standard authentication rules for the
system. I'm not sure if it's worth doing or if it would create more
issues than it would resolve.

Maybe guidance to users to simply not install the documentation packages
on "family" computers would be more appropriate. Of course that would
mean there are applications with no associated man pages installed on
the system....

> and to leave the rhetoric about why this is a wonderful
> package and how it will change the world to languish in the description
> field of the debian/control file.

Most of that is on the websites at sourceforge.net and launchpad.net...

Cheers
Beaky.


--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@kennedy.bevteccom.co.uk
Andrew McMillan
2010-02-19 11:09:15 UTC
Permalink
On Fri, 2010-02-19 at 09:13 +0000, Beaky, The WACS Man wrote:
> Andrew
>
> On Fri, 2010-02-19 at 13:59 +1300, Andrew McMillan wrote:
> > Why would you want to restrict local users from reading files which are
> > freely downloadable?
>
> Generally the act of downloading is a significant barrier to entry for
> many people, particularly those who are not normally the "administrator"
> of the system.

I think the point I was missing here was that the documentation for the
system is X-rated. You haven't said so, but I guess a nod's as good as
a wink, eh?

Perhaps some obscurity helps, and so you might want to write a lintian
override for that - I guess it would slow my 12 year old down enough
that he'd get distracted by something else, though I doubt it would stop
him if he wanted to find it.


> Not really my intention. Just trying to explain that our motivations
> for working on the package are very much the same as for the rest of the
> free software movement. I've found over the years that merely because
> of the subject matter people seem to become suspicious of ulterior
> motives when there really aren't any.

That's OK. I don't doubt your motives, but I also don't expect you to
find too much enthusiasm among this community. Still: you only really
need to find one enthusiastic person, I guess.


> I do see the permissions issue as one of the biggest. It's also what
> lintian has the biggest problem with, and therefore the (technical) one
> that is most likely to need consideration along the road to (hopefully)
> acceptance.
>
> I'm really interested in other people's views on whether protecting the
> system components in this way is a reasonable precaution to take or an
> unacceptable restriction on the other user's rights. I can see both
> arguments.

Regardless of whether the content is X-Rated, my personal feeling is
that protecting it from local users just because they *might* be
underage is overdoing it. Such decisions are normally considered the
right of the person administering the computer, and I'm not sure that
should be any different in this case.

What would be an acceptable compromise, perhaps, would be to install the
files with normal permissions, but ask a debconf question during the
installation, offering to apply a more restrictive umask.


> I'm thinking about protecting the /usr/share/wacs/docs/ directory with
> a .htaccess file that requires a username and password in the upcoming
> 0.8.5 release. That would partially remove one potential path to
> circumvention of the group permissions protection, but it would be hard
> to tie that into all of the standard authentication rules for the
> system. I'm not sure if it's worth doing or if it would create more
> issues than it would resolve.

Indeed, it seems like an approach that could be more hassle than it's
worth.


> Maybe guidance to users to simply not install the documentation packages
> on "family" computers would be more appropriate. Of course that would
> mean there are applications with no associated man pages installed on
> the system....

For web applications it's not unusual that man pages aren't present, but
I assume there are a bunch of command-line tools you're referring to
here.

Would it be possible also to split the documentation into two packages,
so that the NSFW docs weren't included along with the man pages?

Cheers,
Andrew.

------------------------------------------------------------------------
andrew (AT) morphoss (DOT) com +64(272)DEBIAN
You love peace.
------------------------------------------------------------------------
Beaky
2010-02-19 18:28:48 UTC
Permalink
On Sat, 2010-02-20 at 00:09 +1300, Andrew McMillan wrote:
> I think the point I was missing here was that the documentation for the
> system is X-rated.

Not exactly. Inappropriate content in screen shots is blacked out, but
the concepts discussed in the text and illustrated in the icons most
definitely are heading that way.

> Perhaps some obscurity helps, and so you might want to write a lintian
> override for that - I guess it would slow my 12 year old down enough
> that he'd get distracted by something else, though I doubt it would stop
> him if he wanted to find it.

Understood. As far as I'm concerned, that's an acceptable outcome.
It's not about hiding that which can be found in an open subversion
repository, but more about discouraging casual curiosity and having at
least considered the option and raised the bar to entry.

> Regardless of whether the content is X-Rated, my personal feeling is
> that protecting it from local users just because they *might* be
> underage is overdoing it.

Generally the norm in our society seems to incline towards the assume
the worst strategy and so I am generally inclined to follow that as the
default, but leave the choice to open it up available to those who want
to change it. The restrictive permissions aren't really a functional
impediment anyway for normal use as it's primarily GUI resources, perl
modules and stylesheets that would be accessed through a web browser
anyway.

> What would be an acceptable compromise, perhaps, would be to install the
> files with normal permissions, but ask a debconf question during the
> installation, offering to apply a more restrictive umask.

I'm a little loathed to do a packaging specific exception here as it
would make it different from the Fedora process (which does not allow
interaction with the package during installation); it would make more
sense to ask those questions in the web-based installer.

> Indeed, it seems like an approach that could be more hassle than it's
> worth.

OK, that's reassuring. I'll go instead for the "don't install the
documentation packages on family computers" advice in the installation
manual then.

> For web applications it's not unusual that man pages aren't present, but
> I assume there are a bunch of command-line tools you're referring to
> here.

The manual pages are basically new at this release (the CLI interface is
also documented in the configuration and user manuals) and I've not
actually yet incorporated them into the packaging control files yet.
Whether they go in the same package as the commands they describe or in
with the documentation packages is totally open at the moment.

> Would it be possible also to split the documentation into two packages,
> so that the NSFW docs weren't included along with the man pages?

At present the docs are two packages, one containing the PDF versions,
the other containing the HTML versions. The manpages are new and have
yet to be allocated to a package.

Cheers
Beaky.


--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@cyane.bevteccom.co.uk
sean finney
2010-02-19 06:48:04 UTC
Permalink
hi,

On Thu, Feb 18, 2010 at 11:24:40PM +0000, Beaky, The WACS Man wrote:
> Sean

> > it to another list). but at the same time, please no more links
> > to "example" sites!
>
> Sorry, I am not intending any upset.
>
> The example site is completely genuine, only contains material licensed
> under the creative commons by attribution 3.0 license and is actually
> running the WACS software and basically nothing else. It is not some
> devious front for a commercial organisation I assure you. It is 100%
> genuine and true to the ideals of free software.

I'm sure it is... but like i said i'm not making any moral judgement calls
here, nor am i questioning your authenticity as a member of the free software
community. however, i don't want "adult" url's (okay, in your case it was
just a domainname, but still) appearing in the list archives and possibly
endangering the standing of the machine hosting it (which also hosts
version control repos, etc) when it comes to search results, content
filtering proxies, etc. it also distracts the topic away from any
actual technical issues that you might want to discuss. can we leave
it at that?

> OK, then it's obviously not quite as I was led to believe, but that's
> fine, just let me know who I should be talking to. I do however feel
> that due to the nature of the subject having a second person go over it
> will be distinctly advantageous in this case.

yes it probably would. like andrew and i have said, try on debian-mentors
too, as there's a larger pool of people there who might be willing to sponsor
the application. you can also read up on the debian wiki about becoming
a "Debian Maintainer" (different from "Debian Developer") to take over
official maintenance of the package after you find a sponsor.

> For instance, as things stand, I protect almost all files with mode 660
> to the group of package administrators which causes lintian to get very
> upset. I think that is the right thing to do and that the icons,
> thumbnails and documentation should not be available for all users on
> the system to read. That obviously runs counter to the normal
> permissions mentality reflected in the lintian process.

it does seem a bit wierd to have files 640/root:www-data, since the users
could just fetch the URL's instead, or download the deb and extract the
files themselves. if you're talking about *uploaded* content i think that's
a different matter though.

> Given it's one of the biggest markets on the internet, I'm surprised
> that no one has addressed this arena with free software previously (at
> least not to my knowledge). I know a great many people sweep their
> activities in this area under the carpet, but I really hope someone will
> have the courage to come forward and help.

i'll just echo what andrew said.


sean
Thomas Koch
2010-02-20 08:18:26 UTC
Permalink
Hi Beaky,

I've absolutly no problem with adult content as long as the models don't have
any problems producing it.
But I've problems with inappropriate software design. From a first glance at
your code I find for example trunk > manage > wacssetmgr:
http://wacsip.svn.sourceforge.net/viewvc/wacsip/trunk/manage/wacssetmgr

<code>
89 print "<h1 align=center>ERROR:</h1>\n";
90 print "<h2 align=center>Set Number ".
91 $cgihandle->param('setno').
92 " Not Found.</h2>\n";
93 print "<p>\n";
94 print "<center>\n";
95 print "<a href=\"".conf_get_attr("server","wacsmain").
96 "\">";
97 print "Back to WACS Main Menu</a>\n";
98 print "</center>\n";
</code>

- mixing of program code and presentation
- bad HTML style:
- align=center misses quotes
- it should better be style="..."
- it would be much better in a separate CSS file
- smell of code duplication (shouldn't there be a central place to format
error messages?
- No possibility to hook in internationalization
- Five levels of "if" in this file

These are the problems found after less then 5 minutes of search. I've not yet
tested the resistence of the application against common vulnerabilities like
XSS attacks.
Please don't take this personaly, but I dare to question the benefit for
Debian users of having this application in the archive.

No harm intended,

Thomas Koch, http://www.koch.ro


--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@koch.ro
Beaky, The WACS Man
2010-02-21 00:10:52 UTC
Permalink
Thomas

On Sat, 2010-02-20 at 09:18 +0100, Thomas Koch wrote:
> But I've problems with inappropriate software design. From a first glance at
> your code I find for example trunk > manage > wacssetmgr:
> http://wacsip.svn.sourceforge.net/viewvc/wacsip/trunk/manage/wacssetmgr
>
> <code>
> 89 print "<h1 align=center>ERROR:</h1>\n";
> 90 print "<h2 align=center>Set Number ".
> 91 $cgihandle->param('setno').
> 92 " Not Found.</h2>\n";
> 93 print "<p>\n";
> 94 print "<center>\n";
> 95 print "<a href=\"".conf_get_attr("server","wacsmain").
> 96 "\">";
> 97 print "Back to WACS Main Menu</a>\n";
> 98 print "</center>\n";
> </code>

OK.

> - mixing of program code and presentation

... is reliable - back when it was written extensive use of tables and
inline formatting was the only way to be sure it worked well with IE5
and IE6 which were then the current versions of IE. They are still the
versions used by a massive proportion of the internet community.

You may not realise this but the tool you've picked on is a collection
management tool only available to collection administrators. It's also
one of the oldest pieces of code in the system.

> - bad HTML style:
> - align=center misses quotes
> - it should better be style="..."
> - it would be much better in a separate CSS file

Not if you want it to work with old IE versions and don't have the time
and patience to debug endlessly with IE on a Windoze box where CSS is
next to useless. I find about 2 minutes of trying to use windows puts
me in the mood to try violent defenestration.

> - smell of code duplication (shouldn't there be a central place to format
> error messages?

There may well be code duplication. Over time routines have been
standardised and improved, but not all the tools have caught up with it
yet. There is a huge codebase to the WACS project.

We've been focusing on functionality and delivering a complete and
functional system.

> - No possibility to hook in internationalization

Not something I've ever looked at. Either myself or the other developer
with commit privileges will be more than happy to accept patches.

> - Five levels of "if" in this file

If you look at the deltas over the last month or so, you'll see that
some of the oldest of the end user presentation apps, in particular, the
all important model pages have been re-written to significantly reduce
these high levels of conditional code. This is a massive multi-year
project - it's still most definitely a 0.x release series and there is
still a lot of work to be done.

> These are the problems found after less then 5 minutes of search. I've not yet
> tested the resistence of the application against common vulnerabilities like
> XSS attacks.

There may well be problems in that area. We have code in the API (the
makedbsafe function) that handles that and is being progressively
integrated across the functions. Right now I believe we're probably 90%
of the way there on end-user visible applications and maybe only 40%
there on the collection management tools which are locked down to
administrative users from known IP addresses only and so pose less of a
threat. Most production internet sites would not even install them -
there's a discussion of these issues in the Installation Guide and again
in the Administration Guide.

> Please don't take this personaly, but I dare to question the benefit for
> Debian users of having this application in the archive.

You seem to be implying that you think that they'd honestly be better
off with nothing or having to pay tens of thousands of dollars for a
proprietary alternative with less functionality. That seems to me an
incredibly odd attitude for a free software developer; I'd really
appreciate an explanation of the rationale here because I certainly
don't see it.

If you can show me something better, or with even half the capabilities,
then fantastic - believe me, I and the other developers would not be
working on it if we could just use some other package already out there.
To the best of our knowledge, and after much research, we think we can
say that there is absolutely nothing out there in the free software
world in the same arena.

Only very expensive, less capable, proprietary packages. I was
absolutely amazed what some companies were asking for even the simplest
tools to build thumbnail indexes and gallery indexes. One firm in
Germany was charging over 3,000 Euro per module for tools for each of
those simple activities. I've been in the computer industry a long time
and thought I'd seen most of the tricks that could be played but I was
truly appalled at how much was being asked for software providing so
piteously little functionality.

> No harm intended,

But you are causing harm if you seek to block out a package that fills a
need and offers a free software solution to oppressed users, merely
because it isn't coded in what you consider the finest style. That has
to be one of the worst reasons ever to say no.

If you're not going to help them, you might at least not stand in the
way of others who are trying to do so.

Why not volunteer some of your time to help us clean up the code base
and make the package better? I'd welcome the help; honestly we all
would.

WACS is a very big package by now - it consists of over 60,000 lines of
perl code and well over 10,000 lines of PHP code. There are also nearly
five hundred pages of documentation. It is the result of over six years
work by about half-a-dozen software developers.

Functionality and feature-wise it easily betters the very best web sites
in the industry. The in-house software of the ATK Group is probably one
of the closest in abilities (it's the only other one I know of that can
search by Photographer for instance), but I very much doubt that is even
available on the open market, much less within the financial reach of
ordinary users or small commercial sites. Others like the sites from
Pulsar Media have better Location search capabilities (although not
two-level like ours in WACS) but lack the photographer search. Again
I'm not aware of their hosting software being available for sale either.

So come on... come over, give us a hand cleaning up this code and let's
make sure that we give the users and small businesses working in one of
the biggest industries on the internet the chance to see that the free
software community can help them too.

Cheers
Beaky


--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@kennedy.bevteccom.co.uk
Thomas Koch
2010-02-21 08:17:01 UTC
Permalink
Beaky,

I've been a bit harsh. However, since I'm currently doing mostly PHP and web
development I'm faced day by day with "just-good-enough" "solutions".
And it annoys me. These kind of solutions mostly solve your current problem
quickly and without much investment. However, once you've installed them and
want to add additional functionality you get hit by the poor design of the
software.
Then poor web developers get forced to spend hours hacking spaghetti code and
making the whole thing worse with every added feature.

Yes. I think, many times it's better not to have anything at all instead of a
bad solution that gives you the false impression of a solved problem.

However I think, that your software may serve as a good prototype. I'm just
not convinced, that users benefit from its inclusion in Debian:

- One version of the software must be maintained (security bugfixes) during
the whole lifetime of a Debian stable release, which may be something between
2,5-3,5 years. The upstream code will most likely change a lot during this
time, however the Debian maintainer still must fix years old code.

- Once Debian moves forward to a new stable version, you must provide the
Debian stable users with an easy migration path to the newest version. The
upstream code (database schemes) may have made several migrations during this
time, which you must somehow script in one version upgrade path.

- Users that want to try your software and just issue an apt-get install wacs
will get an outdated version of your software and may not realize, that you've
a much better, newer version on your webpage.

So it's not only arrogant snobism about software quality from my side but it
may also be in your very own interest, not to invest in maintaining a Debian
package at this time. Remember, that the first two points from the list above
require manpower, that is therefor not available for upstream development.

I've once been involved in eGroupWare, which gained most of it popularity from
it's very easy automatic web installer:
- download, untar in your web servers dir
- open http://yourserver/install.php

You'll gain much more user satisfaction from such an installer then from a
Debian package. Such an installer also works for Windows, Mac and other Linux
distros. Once this installer gained you a bigger user (and developer)
community and your project has matured enough, somebody from the developers
will step and package it for it's distro of choice.

Therefor it's not a requirement, but a very good indicator, if the upstream
developer and the Debian packager are distinct persion: It means, that the
project is popular enough, that somebody else then the creator had the urge to
package it.

Best regards,

Thomas Koch, http://www.koch.ro


--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@koch.ro
Beaky, The WACS Man
2010-02-21 11:14:39 UTC
Permalink
Thomas

On Sun, 2010-02-21 at 09:17 +0100, Thomas Koch wrote:
> However, since I'm currently doing mostly PHP and web
> development I'm faced day by day with "just-good-enough" "solutions".
> And it annoys me. These kind of solutions mostly solve your current problem
> quickly and without much investment. However, once you've installed them and
> want to add additional functionality you get hit by the poor design of the
> software.

This is where I think you're not really appreciating the way people are
expected to deploy this application in commercial use. We expect people
to create their websites using either the template site we provide as a
basis, or by designing their own pages and then merely adding the WACS
API calls into what they've created in Dreamweaver or whatever.

If you take a look at the Wacs-PHP Simple Skin you will find a set of
PHP end-user presentation apps that are significantly implemented in the
way you suggest. The content and the styling are totally separate and
the CSS file laid out in such a way as to make relatively easy to style.
It's extensively commented to try to make it easier for a web programmer
to understand what is going on.

There's a very significant 120 page Programming Guide that includes a
step-by-step tutorial, a full API reference manual detailing all the
calls, return value and purpose, and a full database schema along with
guidance on the predefined values that are implicitly expected by the
underlying code. There are also various tutorial examples that show how
code using the API is progressively built up.

Web developers are dealing with a straight forward, thoroughly
documented API provided in two different languages but with (almost)
exactly the same semantics.

> Then poor web developers get forced to spend hours hacking spaghetti code and
> making the whole thing worse with every added feature.

I'd be very surprised indeed if most developers using WACS would ever
get involved with the basic toolset code. It's not part of the end user
delivery chain at all. It's merely there to provide an administrative
GUI to the underlying database.

> Yes. I think, many times it's better not to have anything at all instead of a
> bad solution that gives you the false impression of a solved problem.

Understood. I think you're getting a false impression of the package by
focusing on the back-of-house collection database management tools which
are not part of the delivery chain.

> However I think, that your software may serve as a good prototype.

Absolutely. Effectively that is what it provides from the viewpoint of
the web developer. It's totally modular. Other than the database
schema and the API, there's nothing that you have to use. You can
maintain the entire collection using SQL or the command line tools if
you want to. Everything to enable you to do that is described in the
documentation.

If you don't like the Web GUI for database maintenance, you can use the
CLI instead. If you don't like the CLI, you can use SQL and the shell.
The architecture is both open and extensively documented.

> I'm just not convinced, that users benefit from its inclusion in Debian:

Users find software through packages listed in their distribution's
catalog. Very, very few seek it outside of that framework. Those that
do find themselves with all kind of version/revision issues and
problems.

Past and present experience of the support requests for WACS show very
clearly that it is the subtle differences between distributions that
cause well over 90% of the problems for users. The best way to address
that is to make the package part of distributions so that the users can
be assured that the two will work together. Right now we directly
support and provide packages for the current releases of Fedora and
Ubuntu and we're working on getting the package into those distributions
too.

> - One version of the software must be maintained (security bugfixes) during
> the whole lifetime of a Debian stable release, which may be something between
> 2,5-3,5 years. The upstream code will most likely change a lot during this
> time, however the Debian maintainer still must fix years old code.

Absolutely - that is one of the major points of being a Linux
distribution. This is one of the reasons why we've not been talking to
the Debian project about this package until six years into the
development of the project. We wanted to make sure that we had a
credible level of maturity and stability before we made these
approaches.

> - Once Debian moves forward to a new stable version, you must provide the
> Debian stable users with an easy migration path to the newest version. The
> upstream code (database schemes) may have made several migrations during this
> time, which you must somehow script in one version upgrade path.

Yes, that is one of my problems as a developer. We have our first major
schema changes in nearly three years coming up in WACS 0.8.5, and none
of those will be mandatory until at least the WACS 0.9.x series. We
always introduce schema changes a full release cycle before we make use
of them - that's always been policy.

We will be providing a migration path or making sure the code functions
correctly with the old schema, but that will not be strictly relevant
for the Debian packaging of WACS 0.8.5 or later as it is unlikely to
change again for some time. Never-the-less, the commitment is there.

> - Users that want to try your software and just issue an apt-get install wacs
> will get an outdated version of your software and may not realize, that you've
> a much better, newer version on your webpage.

Yes, this is situation normal. But if the older release works for them,
then that is fine. We're making this approach now because we're pretty
confident that the WACS 0.8.5 baseline that we're releasing next month
will form the basis of the platform for the foreseeable future. The
schema changes that this release incorporates are the condensed result
of issues our development work has highlighted over the last three
years.

> So it's not only arrogant snobism about software quality from my side but it
> may also be in your very own interest, not to invest in maintaining a Debian
> package at this time.

I do not feel it is appropriate for me to maintain the Debian version of
this package. I thought I made that clear at the outset. There are
sufficient considerations due to the content that what is included and
what is not should be subject to the checks-and-balances of being
reviewed by someone else. I would consider the debian community very
ill-advised if they expected me to do this work.

I've been too close to the adult industry for too long to consider my
perspectives on acceptability to reflect those of all the members of
society that Debian might wish to engage with.

> Remember, that the first two points from the list above
> require manpower, that is therefor not available for upstream development.

Understood. Right now, WACS is heading from a rapid development cycle
into a more stable release cycle with fewer changes expected. This is
why we're choosing this moment to work on getting packages into
distributions where we can.

> I've once been involved in eGroupWare, which gained most of it popularity from
> it's very easy automatic web installer:
> - download, untar in your web servers dir
> - open http://yourserver/install.php

If only that would work. The reality is that our extensive dependencies
make it virtually impossible to auto-install on a new platform that we
haven't seen before.

This morning I've had a bug report that indicates that Debian Lenny
doesn't successfully run our Ubuntu 9.10 .deb packages, whereas it did
on run just fine on previous Debian releases.

We're obviously working on getting a fix for that user but it just
emphasizes to me how important it is that we integrate tighter with the
distributions and don't leave it to the end users to bear the brunt of
distribution upgrade incompatibilities. Anyone doing even basic testing
on Debian Lenny would have found the problem (it's IPv6 related).

However as a software developer I can't track every release from every
distribution, and without the distribution providing us with support in
terms of testing, indexing and promoting our application, the reality is
that fewer distributions will be supported and more users will hit
distribution related problems.

> You'll gain much more user satisfaction from such an installer then from a
> Debian package.

Unfortunately this has not proven to be the case - it has caused more
user dis-satisfaction than all other parts of the system put together.

Initially we had a generic installer based approach, but have been
forced to relegate it to port of last resort because the dependency
matrix is just too complex. At this point I am considering removing the
installer completely as I think it's unlikely to be able to resolve all
the issues it encounters on a new platform.

> Such an installer also works for Windows, Mac and other Linux
> distros.

Not a snowball's chance in hell, unfortunately. We used to support Macs
but when we found that the dependency build process was taking more than
six hours to complete and that that caused a hardware failure on our
MacBook due to overheating, we reluctantly had to drop support for Macs.
We did not want to endanger user's computers and packaging up all the
components we needed at the revision levels we needed was taking too
much time away from core code development.

> Once this installer gained you a bigger user (and developer)
> community and your project has matured enough, somebody from the developers
> will step and package it for it's distro of choice.

That's what we'd hoped initially. The reality has become that we've
gone from believing we were a Linux and MacOS X application to realising
that we're a Fedora 10-12 and Ubuntu 8.10-9.10 application at this
point. Anything else just isn't that likely to work.

Why? The main problem areas are (in order of severity) PAM
Authentication, HTTP server configuration methodologies, IP layer
configuration, Perl and PHP module packaging and package naming
conventions.

> Therefor it's not a requirement, but a very good indicator, if the upstream
> developer and the Debian packager are distinct persion: It means, that the
> project is popular enough, that somebody else then the creator had the urge to
> package it.

And it may be that Debian will have to be placed on the unsupported list
in the same way that MacOS X has been. I don't see that as a desirable
scenario - some people will choose not to use WACS and won't gain the
benefits of using it, others will choose to cease using Debian. Given
the lack of alternatives in the market place if the role to be fulfilled
by WACS is mission-critical to the user, it may well be that Debian
looses out. I don't want to see that happen.

The reality is if I'm investing the time and money in a third supported
platform it will be an Enterprise platform like RHEL or OpenSolaris
where the likelihood of a financial return from a commercial support or
consultancy contract is significantly higher.

So I'm hoping someone here or on the debian-mentors list will decide to
get involved. This is an upstream developer holding out a hand and
saying "come help me and together we can fulfill the needs of a whole
new group of users". If the answer is "no, we don't care" then we all
loose out.

Cheers
Beaky



--
To UNSUBSCRIBE, email to debian-webapps-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Archive: http://lists.debian.org/***@kennedy.bevteccom.co.uk
Loading...