OpenPKG Roadmap =============== OpenPKG Release Schedule ------------------------ o new release every 4 months (= 3 releases/year) o release date is shifted up to +/- 2 weeks o proposed times: Feb, Jun, Nov OpenPKG Milestones ------------------ PAST: o 2001-11 OpenPKG Project Kickoff o 2002-04 official exclusive deployment at C&W o 2002-01-11 OpenPKG 1.0 (including C&W press release) o 2002-08-28 OpenPKG 1.1 o 2002-12 openpkg-tool (RDF index, build tool) o 2003-01-22 OpenPKG 1.2 o 2003-08-04 OpenPKG 1.3 o 2003-09 Upgrade RPM 4.0.2 to 4.2.1 o 2004-02-25 OpenPKG 2.0 (RPM 4.2.1, frontend, 2x platforms) o 2004-07-20 OpenPKG 2.1 (tool chain, slideset, RPM extensions) o 2004-10-20 OpenPKG 2.2 o 2005-02-24 OpenPKG 2.3 o 2005-06-22 OpenPKG 2.4 o 2005-10-19 OpenPKG 2.5 o 2006-06-22 OpenPKG Community 2-STABLE-20060622 o 2006-10-18 OpenPKG Community 2-STABLE-20061018 o 2006-12-12 OpenPKG Enterprise 1 (E1.0-RELEASE) FUTURE: o TBD successor of OpenPKG 2.5 Unscheduled QA Tasks -------------------- o Test to ensure clean deinstallation of all BIN bootstrap packages Unscheduled Development Tasks ----------------------------- o openpkg.ORG website beautification and tech/optical alignment with .NET [rse] o %doc [mk] o Kolab2 integration [kk] o OpenPKG Request Tracking Migration - export data from old RT/PostgreSQL - store data in new mbox format o OpenPKG Buildfarm Software Enhancements - per-machine classes - per-machine package exclusions o Package Problems - Bash 3.0: openpkg-dev:root, BASH_ENV? o Package dependencies - Adjust dated dependencies (amd, aspell, zebra...) o Fixing: bsd3: R_X86_64_32S relocation bug, -fPIC - perl-*, pango o Fixing: eso2: gcc/ld -Wl,-search_paths_first (occurs for openssh when configuring openssl!) o Fixing: bsd3: fsl/l2 fails: va_list l2_util_fmt_dump (makes trouble for ntp, openssh, cvs, etc) o Fixing: bsd4: configure - Gdb ia64-unknown-freebsd5.3 o Fixing: gtk2, gimp, etc. require C++??? o Fixing: eso1: GNU tar 1.15.x build crashes vendor compiler while GNU tar 1.14 doesn't trigger this o Fixing: OSSP uuid broken on 64-platforms o Correct sudo package, ./install-sh: strip: not found o Bootstrap improvements - Add hooks for Solaris10 smf (not using /etc/rc*.d) o Fixing: packages depending on C++ shall require gcc::with_cxx = yes o OpenPKG Repository Browsing WebUI - OpenPKG rdf2sql - OSSP quos (RDBMS query WebUI) o add .spec file feature of remembering SHA1/MD5 & file size for distfile verification o php CURRENT with pdflib activation fails o perl-dbi broken with DBD's mysql & oracle o finally provide anonymous-ssh based access to CVS o RIPE packages and dependencies (mysql3) are released for production use by GNOC o have a generic way to mark hundreds of config files in the RPM filelist (see webmin) o package popularity CGI, to indicate usage and lead to blessing decisions o argument (generic if possible) error checking o - check command line against RPM macros and .spec file option --define variables o - check bootstrap against legal usernames, ids, prefixes, and tags o apache+mod_php+mod_php_imap+mod_auth_ldap auf Solaris o conflicting files are not detected by rpm (php vs. apache+mod_php -> @l_prefix@/bin/phpize) o apache+mod_php should not provide a @l_prefix@/bin/phpize o cvstrac leap year problem correct checkins Feb-29...Mar-15 en4:/e/(openpkg|ossp)/sw Instructions: http://www.cvstrac.org/cvstrac/tktview?tn=290 o Manual Page "openpkg dev" o openpkg dev: release for PRIV/NoSource packages o Contributor Environment: OSSP cvsfusion: main program o Contributor Environment: "openpkg vendor" o Contributor Environment: HOWTO Document o IMAP broken (telnet host 110) on debian (aw) o Test IMAP virtualhost configuration o fsl import latest pcre ??? using currently version 3.9 -> 4.5 ??? o Consulting Fraunhofer: Write "Local CVS Setup" documentation o build time option for suexec-docroot (and maybe others, too) and deployment to www2-MUC1 o design openpkg-register data format and content o design openpkg-register protocol o Central Registry (UUID based) for OpenPKG instances o brainstorm: Tool Chain API o resolve crash of CVS "pserverd" (and fix cvs.openpkg.org) o OpenPKG Testdrive: install OS o OpenPKG Testdrive: establish jail environment o Showcase (Camtasia) o Multiple Choice Test (MCT) for OpenPKG o contraint based relationship/problem resolver o OpenPKG Tool Chain (minimal) o Cleanup/Extend %xxxx section prolog/epilog (overlay feature) o Bugfix: "+" in versions leads to double-installations o New Websites ("META") o fsl and imapd logging [thl] o openpkg-dev for contributors documentation workoff o openpkg-dev replace which by bash type and remove it o Update Java packages and align to current (stable) community standards o Package gmake not using 'use_make' Argument correctly, possibly generic problem o use %docs flag for flagging documentation o packages should provide a %doc section containing OpenPKG-specific informations (like README.Debian for Debian) o mailing list archive with IMAP/NNTP/??? capability o mailing list archive with Web Interface and URI capability o mailing list archive with Web Interface search o new bugdb based on mailing list archive Web/search/URI/IMAP... o replace rt.openpkg.org with IMAP-based hand-crafted solution o rt replacement: evaluate IMAP server and client features o rt replacement: implement mail filter for request tracking o work-off RT tickets of rt.openpkg.org o import old tickets into new bugdb o check all rcService parameters of rc files against package (rc.*) name, and better build test into rclint o Fix bootstrap: vendor coreutils on platforms following new POSIX standard don't provide some options any longer: head: `-1' option is obsolete; use `-n 1' since this will be removed in the future o extend RPM to reject or at least warn on unmatched --define and %option identifiers [ms] o provide --tmpdir option to bootstrap o Clan.{pm,pod} conflict in Perl packages o consider central log for build/install warnings (Matthias Kurz) o improve SA template [thl] o speclint test for shtool install/subst for sed compatiblity [thl] o platform independent binary packages [thl] o coreutils --with legacy hostname conflict SuSE 8.2 [thl] o conditional Conflicts: bug [thl] o openpkg-tool index should use file locking [thl] o opa macro has side effect on PATH order [thl] o mod_auth_mysql contribution [thl] o traditional owner and groups [thl] o test openssh LDAP support [thl] o document/pick queries and unify across scripts [thl] o sign CURRENT packages [thl] o rpm --rebuild leaving zombie sessions open [thl] o catapult RPM product installer [thl] o rpm reports wrong option settings when showing package info [thl] o explain how users can manage private extensions [thl] ??? which software and which release ?? o unixodbc with FreeTDS [thl] o PEAR packages o apache/php: use php.ini to find pear in php package o obmtool log file mess [thl] o feature request for toolchain [thl] o certification for SuSE, RedHat etc. [thl] o rc scripts and quoting [thl] o extend --ignore(arch|os) logic to --ignoretag [thl] o buglet in rc relying on external commands available in PATH [cs] o check OpenPKG binaries (incl. 2.0) for dup files o openpkg-audit: "openpkg rpm" overload o tar -> shar o "openpkg build" work-off TODO items o rc should check whether it needs *and* can use su to avoid nightly error mails from cron in instances that were installed non-root (s_usr != root) o rse: RPM extension: fetch/curl replacement (OSSP fetch) o RPM bug: "plus in release" -> double installs o "openpkg register" o "openpkg dev" should upload src.rpm -> private o add RPM hooks and prolog/epilog sections (. install-info/mkdir override support . additional logging/auditing possibilities . automatic release requires/provides) o RPM --tag foo remember in binary package o lint-spec/*.spec, %build/%install consistent $RPM_BUILD_ROOT removal o docbook: use sgml with xmlcatalog and only DocBook 4.3? o sgml: use xmlcatalog and rename to xml? o optional possiblity to run BIND under restrictive user/group id o vcheck passive FTP support via LWP o dig into problem where file conflicts are not detected o RPM bug: "rpm alias" is broken and requires blanks instead of "=" on --opt=value arguments (was: popt cosmetics: shows --tag=XXX but expects --tag XXX (equal vs. space)) o rse: simplify options (for tools, etc): [rt, cvs] - solution 1: remove all non-binary %options by replacing them with run-time decisions (config files, wrapper, etc.) - solution 2: %option with_xxx binary, %option [use_]xxx arbitrary string - solution 3: %option with_xxx binary, %variable xxx arbitrary string o use UUID for COOKIE header or add additional UUID header o "OpenPKG" package branding with %{l_openpkg_release} o rse: architecture/platform independent packages (BuildArch: noarch, aber BuildOS: any existing nicht) o thl: explicit package documentation: %doc flagging; add knowledge to "openpkg build" for keeping --includedocs option on upgrades (via "rpm -qsd | grep '^installed'"); o "openpkg dev" creates /// in nosrc pathnames o "openpkg dev" should support sane environment (i.e. openldap "ls -t" check fails because ls shell function) o rse: remember bootstrap pre-installation status (/etc/shells exists, link/dir existed, uid/gid existed, ...) o basic set of meta-xxx packages (concept not fully clear and using CURRENT doesn't hurt at all) o cvstraq bug: http://cvs.openpkg.org/timeline?x=1&c=2&dm=1&px=openpkg-src/apache shows both apache and apache2 timeline o perl-openpkg vs. perl-XXX .packlist removal o rpm: provide additional dependencies (for libs) and dep or|and o rpm: remember source and binary dependencies in both source and binary packages for easier indexing o rc: convert priority based deps into logical deps o bugfix: oident does not start -> core o bugfix: delegate segfaults on Solaris o OpenPKG: openpkg.spec: cleanup: uname -> platform o rse: Sanity Check: Prefix: %{l_prefix} (relocatable) -> make sure only /bin/rpm can be used for install, except if --prefix= with /bin/rpm is used o FAQ: explain crontab user root issue o FAQ: explain root/cw build issues o FAQ: mention vserver/jail is required for security, root might help o framework for switch-user during build+install times (idea: "openpkg install foo" checks "rpm -qplv foo.src.rpm" for setuid/setgid/owner and switches to root only if necessary. Similar for -Uvh step) o unified vendor package import/export o closer ZfOS integration (support config management, etc) o OSSP fsl improvement: separated config file parsing, daemon/deattachement mode, etc. o META o WebUI: Package repository browser for www.openpkg.org o Excel-Sheet ins Web stellen! o OpenPKG Toolchain (Execution Wrapper, User Interfaces, Commands, Perl API) - includes/replacement for src2make, etc (RE+buildfarm) - includes registration framework - backend database (for registration & build-farm & repository browsing) - new curses-based user interface o Training Documentation Set - introduction presentation - introduction article - user guide - student guide o set gcc to 64bit on Solaris (howto by default in gcc (patch), are there any real problems?) o global with_debug=yes options for -g (easy if GCC is a PreReq) o write openssl-snakeoil CA o ASCII tar (shell script) o Automatic, enforced registration o Community Forum my.openpkg.org: - RedHat Network (RHN) for users (self-registred) - CVS Feedback forum for vendors (registered by us) - Package Maintainer forum o Optional Support for Dynamic Shared Object (DSO) Libraries (with_dso=yes) o 100% rpmlint (binary RPM checking): man pages for %{l_prefix} o Bug Database replacement o OSSP var: vartool (enhancement for "shtool subst") o Configuration Management (/etc/*: in-place editing, generating) o Daemon Monitoring (run-time checking, automatic restart) o Certification Test (MCT+Practice) - OPCE: OpenPKG Certified Engineer (Bandbreite) (MCT + Practice [download, solve deps, edit configs, .rpmsave, how to apply SA, corrupted file find+fix, ...]) - OPCD: OpenPKG Certified Developer (Spezialist) (MCT + Practice [develop a package]) o Unix/OpenPKG Curriculum ("all-in-one training & certification") o unknown: creation/update/deletion of external things (user, group, crontab, etc) o testdrive.openpkg.org o naming of /etc/... init scripts [cs], thl suggestion: use UUID Unscheduled Documentation Tasks ------------------------------- o Update handbook o Edit user tutorial o Review slideset o Finally sort out which platforms we support OpenPKG Request For Packaging ----------------------------- - [rse] typo3 4.0 CMS - http://typo3.com/ - [rse] otrs - - [rse] xara - http://www.xaraxtreme.org/ - [rse] xorp - http://www.xorp.org/ - [rse] firebird - http://firebird.sf.net/ - [rse] sapdb/mysql-maxdb - - [rse] ingres - - [rse] silc - http://www.silcnet.org/ - [rse] joomla - http://www.joomla.org/ - [rse] gforge - http://gforge.org/ - [rse] bricolage CMS - http://bricolage.cc/ - [rse] dict[d] - [rse] logrotate - [rse] fontforge - http://fontforge.sourceforge.net/ - [rse] libpixman, libsvg, xsvg http://www.xsvg.org/ - [rse] lcfg CMS - http://www.lcfg.org/ - [rse] auto nice daem. - http://and.sourceforge.net/ - [rse] sge - http://gridengine.sunsource.net/ - [rse] zonecheck - http://www.zonecheck.fr/ - [rse] thy - http://bonehunter.rulez.org/thy.html - [rse] dcc - http://www.rhyolite.com/anti-spam/dcc/ - [rse] openpksd - http://openpksd.org/ - [rse] opendx - http://www.opendx.org/ - [rse] stud.ip (http://www.studip.de/) - [ext] bb - [rse] diablo - [rse] imake - [ext] lprng/ifhp - [rse] lxr (http://lxr.linux.no/) - [rse] freemind (http://freemind.sourceforge.net/) - [thl] MAPI Toolkit - http://www.sourcextreme.com/projects/outlook/mapi/ - [rse] m[eta-]cvs - [rse] mon - [rse] nlp - [rse] ntop - [ext] nullmailer - [rse] opengroupware - [rse] openoffice - [cs] pear - http://pear.php.net/ pear-* separate PEAR packages, similar to perl-* - [rse] rmagic - [rse] sablevm - [rse] sieve/perlsieve (for imapd) - [ext] snmp with sendmail_mib - [ext] sodipodi - [ext] templates (possibly part of X11 package) - [rse] texlive - [ext] tuxpaint - [rse] xfree86 - [cs] xlhtml - [ext] xprobe - [ext] apache2 - full support - [ext] tightvnc - client+server - [ext] gpart - http://www.stud.uni-hannover.de/user/76201/gpart/ - [ext] jedit - http://www.jedit.org/ - [ext] sced - http://www.cs.wisc.edu/~schenney/sced/sced.html - [ext] giram - http://www.giram.org/ - [ext] grace - http://plasma-gate.weizmann.ac.il/Grace/ - [ext] sourcenavigator - http://sourcenav.sourceforge.net/index.html - [ext] gnome - http://www.gnome.org/ - [ext] lxrun - http://www.ugcs.caltech.edu/~steven/lxrun/ - [ext] osrtspproxy - http://www.rtsp.org/ - [ext] dante (socks) - http://www.inet.no/dante/ - [ext] freemind - http://freemind.sourceforge.net/ (i second it) - [ext] chaosreader - http://users.tpg.com.au/bdgcvb/chaosreader.html - [ext] libhoard - http://www.hoard.org/ - [ext] libsafe - http://www.research.avayalabs.com/project/libsafe/ - [ext] hp2xx - ftp://ftp.gnu.org/gnu/hp2xx/ - [ext] hylafax - http://www.hylafax.org/ - [ext] mgetty - http://www.leo.org/~doering/mgetty/ - [ext] gaim - http://gaim.sourceforge.net/ - [ext] lavaps - http://www.isi.edu/~johnh/SOFTWARE/LAVAPS/ - [ext] xkeycaps - http://www.jwz.org/xkeycaps/ - [ext] xtetris - ftp.x.org - [ext] xmahjongg - ftp.x.org - [mk] monotone - http://www.venge.net/monotone/ - [mk] xxdiff - http://xxdiff.sourceforge.net/ - [mk] tkdiff - http://tkdiff.sourceforge.net/ Known Issues ------------ o Debian v3.1 ndbm trouble - fact: the "sarge" release will not include a ndbm. We have applications that still require the ndbm API. Our choices discussed were: - include ndbm in OS: not possible. We are not involved in Debian release engineering - drop support for OS not offering ndbm: not acceptable. According to our primary sponsor's lead engineer UNIX, Debian is the #1 important Linux distro for them because it allows machines to be setup once, run and being maintained for a long time without having to reinstall the OS. Support for this platform is mandatory. - drop support for application requiring ndbm: bad idea as apache 1.3 mod_auth_xxx is one them. - port applications to not use ndbm: not acceptable as a quick hack. Although a good long term goal we are too deep into the release engineering process to accept that additinal workload which has a completely unforseeable scope. - provide ndbm for OS: fixing/enhancing the OS itself is beyond the scope of OpenPKG. - use OpenPKG gdbm with ndbm support this is the easy way to go for fresh installs that use OpenPKG applications only. That's why we picked it, see "ndbm" section in upgrade.txt. However, installations mixing vendor and OpenPKG stuff and existing installations upgrading might run into trouble. The reason is that gdbm::with_ndbm supports a ndbm API, makes the build process of the application happy and allows them to install and run. But the gdbm::with_ndbm file format on disk is very likely different from that of the vendor's ndbm implementation. Upgrades from OpenPKG v1.3 will have used the vendor ndbm previously. Now they use gdbm::with_ndbm. Any damage can happen, from destroyed ndbm files to appliation crashes to application malfunction (i.e. apache mod_auth_xxx unable to read old ndbm and accidentally grant access). Both fresh installs and upgrades might run into trouble when they mix vendor and OpenPKG software, i.e. use a vendor password creation/maintenance tool which writes vendor ndbm files and use OpenPKG v2.0 application which reads gdbm::with_ndbm file format. Upgraders have two options: 1.) build gdbm with_ndbm=no and build apache with_gdbm_ndbm=no. This reverts to the old behaviour of using the vendor ndbm and, of course, only works on OS that provide one. 2.) convert/recreate your ndbm databases. o Debian v3.1 install-info causing info.dir trouble - fact: the "sarge" release includes a install-info application which works a bit different than the install-info applications on almost any other OS. They claim their application saw the world first and assume the rest of the world will change and become compatible, work around that issue or eliminate their name duplication. The install process of applications detecting presence of install-info but assume the non-debian way leads to two known problems: 1.) a "info.dir" will be created. That file will make it into the RPM package and later inhibits package installations as file conflicts are detected when subsequent installs also provide that file. 2.) when the build process is run as root, it alters data below /var on the build host. When the build process is run as non-root, it might fail at that point. Our choices discussed were: - include "common world" install-info in OS: not possible. We are not involved in Debian release engineering. - drop support for OS not offering "common world" install-info: not acceptable. [see ndbm issue] - drop support for application requiring "common world" install-info: not acceptable. Many prominent CORE and BASE applications are affected. - port applications to use "Debian install-info": bad idea to port common world to the exception, no matter if the exception is right or wrong. - provide "common world" install-info for OS: fixing/enhancing the OS itself is beyond the scope of OpenPKG. It would also break a huge number Debian packages. - use OpenPKG install-info not acceptable as a quick hack. We tried and we believe it is possible. In the end it's a matter of overloading existing vendor utilites (i.e. cp|mv|rm|chmod|chown|chgrp) with OpenPKG tools. This is our prefered solution as it will fix other things like chown(1) interception on "make install". However, we simply gave up as the complexity of the changes are huge. In the end, it just a matter of setting PATH correctly at the correct time. While evaluating the latter, we found inconsistencies in RPM internals which have to be cleaned up to allow implementation of hooks which eventually provide overload functionality. We must assume that such changes will have lots of side effects which take months to clean up. REJECTED SCOPE CREEP. - workaround the requirement to have OpenPKG v2.0 build binary RPMs on Debian v3.1 is to have the Debian install-info either not installed at all or temporary replaced by /bin/true, i.e. using: # mv /usr/sbin/install-info /usr/sbin/install-info.debian # ln -s /bin/true /usr/sbin/install-info o more accurate version.release requirements - decide whether *-2.0.0.(src.)rpm should require: openpkg-2.0.0-2.0.0 or not and why (not) currently all release packages require other packages unversioned, i.e. the relationship of application A requiing library L is formulated as A requires L This is good as long as the user stays within a release. However, when the user starts to mix in CURRENT or is in the progress of doing an UPGRADE it would be favourable when the requirement is formulated: A requires L >= V.V.V-R.R.R A requires L >= *-R.R.R This would ensure L is upgraded before A, something we cannot assure today. The first form doesn't work for mixing in CURRENT beyond the first vendor upgrade. It also doesn't work for doing an UPGRADE assumed the later release also contains a later vendor version. The second form doesn't work because RPM doesn't support it. A possible workaround was identified to omit the vendor version: A requires L >= L.release.R.R.R and L-V.V.V-R.R.R provides L.release.R.R.R This means that every application A and every library L needs to have this information added. This is a maintenance nightmare and, if done manually, considered being a very error prone task. Ralf tried to automate it using macros but failed because the macro did only get information from the last requirement entry not all entries and conditional entries seem to be a even harder challenge. So we stick to what we had in the past and this is deferred due to time constraints. REJECTED SCOPE CREEP o NetBSD - mysql* needs PTH but NetBSD 1.x doesn' support by default