NAME CPAN::Forum - Web forum application to discuss CPAN modules SYNOPSIS Visit DESCRIPTION This is a Web forum application specifically designed to be used for discussing CPAN modules. At one point it might be adapted to be a general forum software but for now it is released in the hope that people will help improving it and by that improving the site. Features * Posting only by authenticated users. * For registration a valid e-mail is required. * Username and password should be in lowercase and unique. * Every poster will have to give 1. name of the group 2. Subject 3. Content 4. (Future) A unique id to the post and the post made in response to. This reference id will be NULL for new posts. At the beginning responses will have to maintain group but can change the subject and will have to write new text. * Later we might enable changing the group to a related group. (e.g. a message about a module can get a response in a group to which this module belongs to) * Later we might enable total random change in the groups. * We make sure the links are search-engine friendly - /posts/ID (link to a post) - /threads/ID (link to a thread) * We provide RSS feed of the recent posts belonging to any of the groups. * We'll provide search capability with restrictions to groups. Authentication Shared authentication with auth.perl.org? I once tried to do this but then for some reason I could not finish the process. Maybe later we'll want to enable our users to use their auth.perl.org identity. Maybe we can do it also with PerlMonks. Right now we have our own registration and login mechanism. INSTALLATION Apache This is the configuration of my Apache server on my notebook AddHandler cgi-script .pl ServerName cpan.local DocumentRoot /home/gabor/work/gabor/public/dev/CPAN/www/ ScriptAliasMatch ^/(.*/.*) /home/gabor/work/gabor/public/dev/CPAN/www/cgi/index.pl/$1 DirectoryIndex cgi/index.pl Options Indexes FollowSymLinks ExecCGI AllowOverride None Order allow,deny Allow from 127.0.0.1 hosts For local installations in /etc/hosts I added: 127.0.0.1 cpan.local That way, I have a totally separate virtual host just for this application. In a real setting probably you'll have something like www.cpanforum.com pointed to your server. Install the perl code perl Build.PL ./Build ./Build test ./Build install dir=/path/to/install cd /path/to/install chmod a+x www/cgi/index.pl (needed only if you work out of the repository) chmod a+x db/forum.db (or whatever you need to make sure the database is writable by the web server. Finally, manually edit the www/cgi/index.pl file and set the sha-bang to the correct one Setup the database In the directory where you installed the modules create a file called CONFIG (see t/CONFIG for an example). Having the following fields: username= The user name of the administrator. email= The E-mail of the administrator. password= The password of the administrator. from= The Email address to be used as the from address in the messages sent by the system. You will be able to change all these values later from the web interface but we need to have the first values. Run: perl bin/setup.pl (you can now delete the CONFIG file) Run: perl bin/populate.pl (this will fetch a file from www.cpan.org and might take a few minutes to run) CPAN_FORUM_URL For some of the tests you'll have to set the CPAN_FORUM_URL environment variable to the URL where you installed the forum. Changes v0.09_05 - POD cleanup (Shlomi Fish) - More tests - Start using Parse::RecDescent v0.09_04 - Before writing a new post instead of showing a list of all the modules now the user first will search for a module name. post link should give a search box that will let the user search within the names of the modules. The result should be a restricted list with only a few module names in a pull-down menu like we have now. The search is a regular SQL LIKE search and we add % signs at both ends of the typed in word. TODO - Decide on Basic Markup language and how to extend for shortcuts opening tag for code: ]*> but right now only should be accepted closing tag for code: - check all submitted fields (restrict posting size to 10.000 Kbyte ? - Make the site look nicer (HTML and css work) - Improve text and explanations. - Improve Legal statement, look at other sites. clean up documentation add indexes to the tables ? show the release dates of the various versions of a module so it is easy to compare that to the post. Authentication and user management process: - new user comes to our site we give him a cookie, when he wants to login we offer him -- login using the auth.perl.org credentials -- login using XYZ credentials -- create local credential -- For auth.perl.org --- redirect the user to auth.perl.org wait till he logs in there (maybe even creates the new account) --- sets the preferences --- comes back --- we can fetch some of the information from that user --- we need to keep the user_id received from auth.perl.org for later identification of the user --- while we tell the user we would like to get the username/fullname/e-mail address from auth he might not want to give, for this case we should have our way to update the locally updated username, full name and validated e-mail address. --- For XYZ we have to see how they work -- For local credentials we need the user to give us username/password/fullname and validated e-mail address. We have to make sure that usernames which are displayed don't collide. Maybe we should use separate fields for usernames from various sources and when displayed we might prefix it auth:gabor, local:gabor etc. Not nice, any better way ? - Add constraint checking to every field that the user can change by submitting information. - Finalize markup Subject field: - <= 50 chars - Can contain any characters, we'll escape them when showing on the web site Text field: - No restriction on line length, let the HTML handle that part - The text is divided into areas of free text and marked sections In order to avoid accepting postings today that will break when we add more tags, we will reject any submission that is not correctly marked up. - Pages: new mesage: EDITOR; PREVIEW + EDITOR show: POST response: POST + EDITOR; POST + PREVIEW + EDITOR thread: P1 + P2 + .. Pn thread response: P1 + P2 + .. Pn + EDITOR; \ P1 + P2 + .. Pn + PREVIEW + EDITOR; When the EDITOR comes up first the subject should be filled by the subject it is answering to or empty for new message. The PREVIEW and the EDITOR should be filled by the same information, though within the editor we don't need the parent id and similar to be shown. - Enable some administrator to mark a message (or a whole thread) to be hidden (database already has field) - Enable some administator to mark a group to be - hidden (messages don't show up) - frozen (cannot add new message but still can see the earlier messages) Critical part: make place for this in database (status field) - Administrator (or even the author ?) should be able to move a message from one module to another module or group. - Enable administrator to ban a user (mark in the users database to disable the user) Hmm, do I really need this ? maybe as I cannot just delete a user. (added a status field that is not used currently) TODO Nice to have - Make sure adding a new module works fine - make paging available responses 1..10, 11.20, etc, OK, so we have listing in places like / /dist/Distro-Name /users/USERNAME /all Can be a name for all the posts so we don't need to put any other information immediately after the first slash maybe it should be /home ? /dist/Distro-Name/start/count /all/start/count We'll also have some search facility that will be a post operation and /posts/id show a post (show post ) /new_post/ start new post ( editor with module list) /new_post/Module start new post ( editor no module list) /response_form/id start a respones (show post + editor) From the forms we have post methods so no need for URL munging process_post => (show previous post)? show editor + show preview TODO Next release only - make the page size (for paging) user configurable - Notify user A user can ask to be notified upon the following events per distribution. subscriptions: uid, gid, (all), (starter), (participate) 1) All messages All the messages execute QUERY: select uid FROM subscription WHERE gid == disto and all. 2) Thread starters Thread starteres (where id=thread) execute QUERY: select uid FROM subscription WHERE gid == distro and starter 3) Followup messages in a thread he participated already Every message (well, except thread starters) execute: QUERY: select uid FROM subscription - there is a post with the same thread id as of this post which was posted by a user which has a subsciption (participate) 4) People who are not subscribed to All messages (1) when seeing an interesting posting they can say: send me all followups. uid, threadid She can set up such notification on a per module basis or for all the modules. After logging in the user can modify his "subscriptions" to such notifications. The notification will be sent out from an e-mail address such as noreply@bla.com which will discard any message sent to it. The message will contain the text of the post, a link to the post_response page, a link to view the whole thread and an e-mail address in case someone wants to complain/whatever. - Subscription (notify) management: - /mypan will be the url to get and set all the configuration information. It will list all your current subscriptions and you can enable/disable them. Normally this will show only distributions you have some kind of a subscription. In addition from /mypan there will be a way to ask to add a new subscription by selecting the name of a module and the initial subscription parameters to it. In addition when displaying the list of all the messages to a specific module, logged in users will see their current subscription to this module (even if that is empty). - Fix Installation - when installing one might need to be root, in order to set the permissions correctly ? - as user www fetch the module list file, unzip it in the db directory (as this is the only directory we can write to) and run the populator - on a new installation, change the ownership of directories (or at leas tell the user to do so) - Write comprehensive test suit - Reply within a thread When replying to a post within a thread we might want to open the editor window in the middle of the thread, just below the post I am responding to. - Make the Session use the database instead of plain file - Make autoposter of new version announcements work A script that will send an announcement on the new version of every module to the list I think this is done as a script listening in the cpan-testers mailing list, though it might be one similar to bin/populate.pl - make sure links that are relevant for distros don't show up on pages which don't belong to distros. (e.g. a link to search.cpan.org/dist/CGI is ok but a link to search.cpan.org/dist/General is not) - Sometime we'll want to post a message in more than one group, e.g. now I'd like to know how to use CGI::Session with DBD::SQLite. I might want to post the message on more than one list at the same time as this is related to more than one module. Porbably if I need to chose one I'll select CGI::Session as I am trying to use that but it might be a nice feature. Maybe I need to tell one module as the main group and then have a way to associate a few more modules with the posting. This can be done by de-coupleing the name of the distribution from the posts table for all the distributions or we can add such an extra table for the additional distributions so there will be a leading distro of the thread. - Getting the listing of all ~7000 module names takes a long time. I should profile it. 1) write a small script that will run the relevant code on the command line, 2) time this 3) look at the size of the output 386K -> it won't fly, you can't have such a page on the web. Other solutions: - type in the name - search for the name - currently we'll keep a separate file called db/modules.txt with a listing of all the distros. This make page generation in 1-2 sec instead of 7-8. Obviously there is a problem we'll have to fix. - Create a group for - each Distribution (DONE) - Some bigger groups (eg. databases, testing, ) maybe put each distribution under one or more of the groups too - General and other special purpose groups such as News (for the site) where only "administrators" can post. I am not sure if I have to keep all these things in one table and if the same form has to serve for creating messages in both distros and categories. - Database or plain files ? I think every information should be in the database but then we might want to generate static pages from the posts and discussions in order to reduce the need to fetch information from the database. Hmm, it sound faster but we'll probabl want to build the pages on the fly anyway so maybe it does not improve anything. We can start off by totally dynamic pages and then see if making them static will reduce the load on the server. First we'll have to have load on the server. :-) - Check if the technique we use to remember the last request before login cannot cause some security problem such as remembering the last request of someone else who used the same machine recently ? - xml - provided - favicon.ico and a banner image would be good Shlomi: The Forum uses cgiapp_prerun to set the mode according to the PATH_INFO instead of using a mode_param code-reference. This causes a lot of warnings in the logs, and doesn't really belong in cgiapp_prerun. It cannot be hosted on a URL except for its own virtual host, as it uses absolute URLs. ("/login/", "/register/", etc.) A better idea would be to track the path that the web-server gives (it's in one of the environment variables) and then to construct a /cpan-forum/login/ /cpan-forum/register/ etc. path. (or use relative URLs). METHODS cgiapp_init Standard CGI::Application method. Setup the Session object and the default HTTP headers setup Regular CGI::Appication method to setup the list of all run modes and the default run mode cgiapp_prerun Regular CGI::Application method We use it to change the run mode according to the requested URL (PATH_INFO). Maybe we should move his code to the mode_param method ? autoload Just to avoid real crashes when user types in bad URLs that happen to include rm=something home This the default run mode, it shows the home page that includes the list of most recent posts. build_listing Receives a CPAN::Forum::Posts iterator and optionally two numbers Builds an array of hashes from all the posts or those in the given range and returns the array reference. redirect_home Just to easily redirect to the home page about About box with some statistics. internal_error Gives a custom Internal error page. Maybe this one should also receive the error message and print it to the log file. load_tmpl Semi standard CGI::Application method to replace the way we load the templates. login_process - Processing the information provided by the user, - calling for authentication - setting the session - redirecting to the page where the user wanted to go before he was diverted to the login page logout Set the session to be logged out and remove personal information from the Session object. _group_selector It is supposed to show the form to write a new message but will probably be a redirection. response_form Probably obsolete. posts Show a post, the editor and a preview - whichever is needed. process_post Process a posting, that is take the values from the CGI object, check if they are acceptable and try to add them to the database. If anything bad happens, give an error message preferably by filling out the form again. threads Show all the posts of a thread. dist List last few posts belonging to this group, provides a link to post a new message within this group users List the posts of a particular user. mypan Planned to be the manager for the notify subscription, currently not in use. search Search form and processor. rss Provide RSS feed /rss latest 20 entries /rss/dist/Distro-Name latest 20 entries of that distro name notify Send out e-mails upon receiving a submission. _text2mail replace the markup used in the posting by things we can use in e-mail messages. ACKNOWLEDGEMENTS Thanks to Offer Kaye for his initial help with HTML and CSS. Thanks to all the people who develop and maintain the underlying technologies. See for a list of tools we used. In addition to Perl of course. DEVELOPMENT Subversion repository is at There is a mailing list to see the commits to the repository: Discussion of this module will take place on If you need help or if you'd like to offer your help. That's the right place to do it. BUGS Please report them at LICENSE Copyright 2004-2005, Gabor Szabo (gabor@pti.co.il) This software is free. It is licensed under the same terms as Perl itself.