OpenPKG Package Master/Servant On Duty (PMOD/PSOD) ================================================== Current Assignments =================== Ralf S. Engelschall (Director R&D) Thomas Lotterer (Assistant R&D) Steve Weinreich (Assistant R&D) Matthias Kurz (Assistant R&D) Week Start Date PMOD ----- ----------- ----- CW-50 2005-12-12 rse CW-51 2005-12-19 thl CW-52 2005-12-26 steve Description =========== Every week there are two dedicated jobs assigned to people of the OpenPKG team: the Package Master On Duty (PMOD) and the Package Servant On Duty (PSOD). The PMOD is the primarily responsible person, the PSOD is his assistant on demand or automatically becomes the PMOD if the PMOD is not available. The people are assigned in advance. If they cannot perform their responsibility, they are liable for reassigning their task to someone other. The daily task of the PMOD is to (in this order): 1. monitor Bugtraq and related forums (see security.txt for URLs) and if we are affected in any way, immediately take action by collecting all relevant information and forwarding them in a reasonably filtered way to the Security Engineers under openpkg-security@openpkg.org. Goal: Take action on security issues as fast as possible. Reason: Security for our user community. 2. monitor the community postings on openpkg-{users,dev,bugdb}@openpkg.org, try to give a reasonable answer or at least let the community know that we are investigating and redirecting the issue to the PSOD or putting it onto the TODO list (openpkg-adm/todo.txt). Goal: OpenPKG community gets responses as fast as possible. Reason: Feedback and warm feeling for our user community. 3. process the bi-daily Version Tracking reports posted on openpkg-team by updating the outdated packages to new versions. New packages are built by the PMOD and copied to master.openpkg.org:/e/openpkg/ftp/current/SRC/. The openpkg-src/foo/foo.spec, openpkg-re/vcheck/vc.foo and openpkg-web/news.txt files are updated and comitted to CVS. If a package is outdated by a new version that is less than of release grade, the out-of-date package is considered superior and left alone. Exceptionally, alpha and beta-grade software is repackaged if the out of date version is alpha or beta-grade also. Goal: Keep OpenPKG-CURRENT up-to-date with minimum delay. Reason: Reputation for being always up-to-date; RelEng in advance. 4. If time permits, additionally investigate in removing issues from the TODO list (openpkg-adm/todo.txt) and our BugDB. Goal: Reduce the pending work. Reason: Obviously more time for new things ;) Working with CVS branches ========================= As shown on page http://www.openpkg.org/releng.html, the OpenPKG CVS Repository contains branch points along the trunk. With few exceptions, the following work process should be followed to keep the CVS Repository consistent. 1. Put changes to the trunk revision of a file first. For example, to work on the trunk revision of a spec file, run: $ cvs upd[ate] -r OPENPKG apache.spec $ vi apache.spec (make changes here) $ cvs com[mit] apache.spec 1a. If the change is minor or unstable, then stop here. For example, the daily task of upgrading 'current' packages, or improving text descriptions belongs on the trunk and not elsewhere. 2. If the change fixes a vendor or packaging bug and is deemed stable, then the change should be applied to the 'stable' branch as well. Sources on this branch are tagged 'OPENPKG_1_STABLE', so use: $ cvs upd[ate] -r OPENPKG_1_STABLE apache.spec $ vi apache.spec (make changes here) $ cvs com[mit] apache.spec 3. If the change corresponds to a security advisory or other serious matter, then it should be applied to the 'solid' branches as well. It is from these branch points that the OpenPKG update packages are made, and placed in the FTP storage under 'UPD'. $ cvs upd[ate] -r OPENPKG_1_2_SOLID apache.spec $ vi apache.spec (make changes here) $ cvs com[mit] apache.spec Consider using CVS itself to merge your changes from one branch to another. If the same changes to the trunk should be applied to the 'stable' branch, then use 'cvs -j 1.22 -j 1.23 apache.spec' in place of 'vi apache.spec' in the above samples. Replace the version numbers and file name of course. To learn which CVS tags a file has, use 'cvs status -v apache.spec'. Also, the tool 'openpkg-dev' has branch logic that acts as an alternative to using the above 'cvs' commands directly. 'openpkg-dev branch OPENPKG_1_STABLE' can be useful when working on a group of files at once. About patching ============== After bumping up the vendor versions, a patch file often does no longer apply cleanly. Then one usually would like to just remove the patch, but... 1. before a patch can be removed as a whole, one has to review the patch as a whole, i.e., all patch parts, and _IN DETAIL_ to make sure that really _ALL_ parts of the patch are obsolete. Often just _some_ parts are obsolete, others would be still valid and required (at least on some platforms). In this case the patch has to be trimmed and adjusted, but kept partly. If the patch is just removed as a whole (simplifies the upgrade task, of course), we loose important patches. It is even not sufficient to just make sure that the package without the patch builds on a single platform. Either you have to check at least all three platforms or _really_ understand the patch in detail. 2. after (1) you still decide to kick out the patch, do it completely. That is, remove the patch from the Patch: header, remove the %patch _AND_ remove it from CVS with "rm ; cvs rm ". Multiple times we had patches flying around in CVS which are no longer used in the current package specification. Issue with vcheck ================= It is fine that the PMODs/PSODs upgrade the packages which the vendor version tracking report successfully identifies for upgrading. Nevertheless we have always multiple packages which were not tracked because of errors. If those packages occur in the report's error list multiple days in sequence, I really would like that the PMODs/PSODs also at least once per week review this. Sometimes it is just vcheck's failure, sometimes the URLs just have changed, etc. So, please treat it as an important part of the PMOD/PSOD task to get rid of this error list, too. Using the tool 'openpkg-dev' ============================ There are two basic operations modes, "developer" and "contributor". A developer must have read and write access to the OpenPKG CVS repository and SSH scp write access to the OpenPKG HTTP and FTP server. He can directly release packages and update any information into the OpenPKG development environment. A contributor only uses read access to the OpenPKG CVS repository. He releases packages by releasing files to a write-only contribution area on the OpenPKG FTP Server. He cannot update any other information, i.e. those on the HTTP Server. A verification script runs quarterly and inspects the upload area. It either drops the upload in case it looks like trash or picks it up and informs developers on the openpkg-dev@openpkg.org mailing list that a new contribution is available for review. Anyone can join the developer mailing list, so contributors can use it to find out if their upload succeeded and passed basic "trash" checks. Anyone can be a contributor. setup a OpenPKG development environment - create personal work directory if not existent $ mkdir ~/work - grab the openpkg-dev tool by either checking it out from the openpkg-re module or download the latest release from the web. Three methods shown here, pick your favourite. $ cvs -d openpkg-cvs@cvs.openpkg.org:/e/openpkg/cvs checkout openpkg-re ; cd openpkg-re $ curl http://cvs.openpkg.org/getfile?f=openpkg-re/openpkg-dev >openpkg-dev && chmod ug+x openpkg-dev $ lynx -source http://cvs.openpkg.org/getfile?f=openpkg-re/openpkg-dev >openpkg-dev && chmod ug+x openpkg-dev - verify the "default configuration" settings in the head of the script. If they don't match your environment, copy'n'paste them to ${HOME}/.openpkg-dev.rc and change them to accommodate your needs. Do not edit the script. It is part of the setup and update procedures to checkout the or update to the latest version of the script and point an alias to this latest version. The example assumes that /cw is your personal OpenPKG playground hierarchy $ vi openpkg-dev ${HOME}/.openpkg-dev.rc ## ## ~/openpkg-dev.rc -- OpenPKG development tool configuration ## example for contributor ## OPENPKG_MODE=contributor OPENPKG_INST=/cw OPENPKG_WORK=${HOME}/work/contributor OPENPKG_TEMP=${HOME}/work/contributor/TMP OPENPKG_REPO=openpkg-cvs@cvs.openpkg.org:/e/openpkg/cvs OPENPKG_DIST=master.openpkg.org:/e/openpkg/ftp/current/SRC/ ## ## ~/openpkg-dev.rc -- OpenPKG development tool configuration ## example for developer ## OPENPKG_MODE=developer OPENPKG_INST= #empty is automatic OPENPKG_WORK=${HOME}/work/pmod-psod OPENPKG_TEMP=${HOME}/work/pmod-psod/TMP OPENPKG_REPO=:pserver:anonymous@cvs.openpkg.org:/e/openpkg/cvs OPENPKG_DIST=ftp://ftp.openpkg.org/contrib/00UPLOAD/ - setup your openpkg-dev environment Four examples are listed below. The shortest example downloads all sources from the CURRENT branch. You can specify a package to avoid downloading all sources which is useful for slow links. In case you don't know the package name, note the examples uses shtool which is useful, appeared in all branches and belongs to the smallest packages. Also note that the re (release engineering) directory always contains all vc (vcheck) files. These are many, but small, so don't be alarmed. Also a branch can be given as a last parameter, the official format is HEAD, OPENPKG_x_STABLE, and OPENPKG_x_y_SOLID. Lot's of aliases and shortcuts are supported, i.e. CURRENT for HEAD, x for OPENPKG_x_STABLE and x.y for OPENPKG_x_y_SOLID. If you want to download all sources and specify a branch, use "" for the package name. Be prepared to download approximately 25MB of data. Unless you kill the environment, this needs to be done only once. $ /tmp/openpkg-dev setup $ /tmp/openpkg-dev setup shtool $ /tmp/openpkg-dev setup shtool OPENPKG_1_STABLE $ /tmp/openpkg-dev setup "" OPENPKG_1_2_SOLID - setup an alias for convenience reasons $ alias opd="$HOME/work/openpkg/re/openpkg-dev" - enter a bash subshell executing the "bash" function to start the actual development. This subshell has enviroment variables set, functions defined and aliases created for "cd", "cvs", "openpkg-dev", "root" and "rpm". Korn shell users might get a preferred vi-style line editing by issuing a "set -o vi" in bash. $ ./openpkg-dev bash Alternatively append the name of the package, the branch to work on and instance to execute rpm from. $ ./openpkg-dev bash [ [ []] Providing all options is equal to $ ./openpkg-dev bash # internal variables exposed to user shell - enforcement, current openpkg-dev$ openpkg-dev package # ${OPENPKG_SPEC}, $P openpkg-dev$ openpkg-dev branch # ${OPENPKG_CTAG}, $B openpkg-dev$ openpkg-dev execute # ${OPENPKG_EXEC}, $E The prompt shows the current values of $P, $B, $E and additional information. A black value [1] indicates it was forced through "bash" "package", "branch" or "execute". Values which were not forced are set by automatic guessing. Even when values are forced, guessing takes place to detect odd situations. Differences between forced values and manual guessing are shown in red [2] However, forced settings take precedence. Executing the "package", "branch" or "execute" commands without parameter clears the enforment and reverts back to guessing. The variables listed above are exported can be used by the user in own scripts or directly on the command line. Guessing takes place with every "cd" command. If external changes have been made to the environment, a "cd ." will clear things. [1] xterm uses black, dumb terminals and xterm window title show a dot '.' after the value. [2] xterm uses red, dumb terminals and xterm window title show a exclamation mark '!' after the value. - update the development environment any time by executing the "update" function. It is useless immediately after setup but should be done often to keep the workspace current. Note that the openpkg-dev itself is located in the re directory that was just checked out. So it updates itself. Update every time before starting development on a package update. openpkg-dev$ openpkg-dev update - development of a new package and the update process is not yet automated, so no braindead copy'n'paste commands here ;-) However, a few useful commands are listed as an example for a typical package update. openpkg-dev$ openpkg-dev track openpkg-dev$ openpkg-dev vim openpkg-dev$ openpkg-dev fetch openpkg-dev$ openpkg-dev build openpkg-dev$ openpkg-dev peek openpkg-dev$ openpkg-dev install # When switching between different OpenPKG releases, i.e. when creating updates, # make sure the dst cache does not contain invalid data from previous downloads! # #OPTION1 remove files for package provided by OpenPKG from dst cache using fetch -s #OPTION2 remove all files for package from dst cache using fetch -a # this is required if vendor uses same filename for different releases - list source and binary packages by executing the "list" function. The package argument is optional and defaults to $P. Multiple packages might be specified. openpkg-dev$ openpkg-dev list openpkg-dev$ openpkg-dev list [[package] ...] - diff before a release by executing the "diff" function. This is the same as adding the "-dry" option to the "release" function. - release a package to the OpenPKG community by executing the "release" function. Members of the OpenPKG development team commit changes directly into the CVS repository and website while contributers upload news and changes to the contributor area. The package argument is optional and defaults to $P. Multiple packages might be specified. Adding the "-dry" option executes a dry run and does not alter anything. For packaging updates and new vendor releases, the commit message is usually generated automatically. If the automatic fails or a manual message should be given, use the "-m" option with an argument to specify a comment for the cvs commit. If a file with name of the argument exists, it's contents will be read to retrieve the actual message. Using '-' as the filename will omit the comment so the cvs command will open an editor. openpkg-dev$ openpkg-dev release -dry openpkg-dev$ openpkg-dev release [[package] ...] [-m "comment"] [-dry] - kill the development environment by executing the "kill" function. This will discard the wholly directory structure and any unsaved data is lost. Since UNIX is a operating system for adults, no questions are asked. openpkg-dev$ openpkg-dev kill