Fink:PackageDatabase:NewDesign

Ideas about Improving the PDB

This page collectes ideas on what could and should be improved in the Package Database (pdb).

Testing Setup

A demo of how the new PDB could look can be found at http://finch.finkproject.org/~chris01/fink_web/pdb/ . The corresponding files are on finch.finkproject.org . The scripts can be found at:

finch.finkproject.org:/home/chris01/new_pdb/scripts/pdb

The files there are checked out from the redesign_pdb branch. And the web pages are at:

finch.finkproject.org:/home/chris01/public_html/fink_web/pdb

also checked out from the redesign_pdb branch. The HOWTO file describes how to clean out the db and start fresh.

The database can be accessed through the phpMyAdmin web interface.

IMPORTANT: Always use the update.sh script to update the cvs checkouts. This scripts fixes the group permissions so that everybody has access to the files. However, for the php files there is no such script. So make sure you do:

chmod -R g+w *

Backend: Implementation issues, Database Infrastructure

In this section, we talk about the "behind the scenes" code of the package database, i.e. the parts not visible to users

General

• Incremental update of package db table. E.g.: (chris01)
  • create a temporary table with new entries
  • Remove everything from old table not in temp table
  • Update entries already in old table
  • Add entries only in temp table
  • and dmacks says: "Now just a minute, why doesn't index.php just do SELECT COUNT(DISTINCT name) FROM package; ?" (chris01)
• Links to the info files in cvs, and last changed date. (chris01)
• Better formatting of the Desc fields (chris01)
• Show info about a specific version, e.g. http://www.finkproject.org/pdb/package.php/cups-dev?version=1.1.15-2(chris01)
  • In the same vein, it should always be clearly visible which version is being displayed -- maybe display that information in the the title, so instead of "Package cups-dev" it shows "Package cups-dev 1.1.20-11"; alternatively, add a "Version:" entry to the table (Fingolfin)
  • The table on package.php which shows the latest known versions of a given package for each dist/tree would probably be linking to these "specific version" pages (Fingolfin)
• exclude virtuals, or handle them gracefully, e.g. by showing all possible virtual packages without a version number, might be difficult (chris01: just don't show them.)
• omit actual locally-installed fink if it is present
• Why are there entries in the old pdb for packages that are no longer in any dist? (E.g. archive-zip-pm586-man) (chris01)
  • They're available only in dists that aren't displayed in the table (flags/"latest" doesn't know releases.inc and package.php only looks at "latest"). archive-zip-pmXXX-man is present in current-10.4-transitional-unstable only. (DMacks 03:22, 30 January 2007 (CET))
  • So we should discuss how we deal with entries of old releases. I suggest: don't show them in the index anymore. Objections? (chris01)
• Show DescUsage (chris01)
• Do we want to hide some information for the default package page, and have an expert mode (e.g. DescUsage, dependencies, CVS, etc.) (chris01)
• Show dependencies? (chris01) And should that include BuildDepends? (akh)
• Fixed a bug where packages were shown in a bindist even though the license was 'Restrictive' at the time of the bindist (e.g. svn-ssl) (chris01)
• How about displaying the output of "fink validate" for every .info file, too? (We would have to store it in the DB, too, for this to work, of course). Benefit: This might coerce some more packagers into validating their packages, as they get a nice shiny green "Package is valid" vs. a list of yellow/orange "Validation warnings/errors". (Fingolfin)
• Should 'htdocs/pdb/db.inc.php' used in header.inc be the same as 'htdocs/db.inc.php'? (chris01)
• Improve selecting splitoffs in package.php (include version info in link) (chris01)
• Make links to splitoffs version/release specific, if package.php was called with version/release parameters (chris01)
• Is "pdb doesn't see the "current" distribution (Fink Bug # 1606084)" solved?

Changes to the DB table layout

• The PDB SQL format is not in the proper form for a relational DB (Fingolfin)
  • fullname is redundant and should go
  • latest and needtest appear redundant, too (Chris01)
  • desclong is preformated, instead of leaving this formatting to the presentation layer
  • "release" table should be refactored:
    • the name "release" for the table / "package" field seems inappropriate. How about changing it to "distribution" ?
    • or rather, it might be split (for the sake of normalization): have a distribution" table, which contains: OS/distro, and priority
    • then, have a "release" (better name?) table, which contains: distro, type (stable/unstable/bindist), version (like "0.8.1" or "current").
    • that way, we have normalized the data, and get rid of releases.inc as it is now trivial to find the stable tree matching an unstable tree and vice versa.
    • however, I am not quite sure how one would now properly "link" packages to their release? Either we keep the current "release name" (somewhat contrary to normalization), or we simply number the releases sequentially, and use that number as key).
    • I suggest the the following DB schema and content for the dist/release tables: Fink:PackageDatabase:NewSchema (Chris01)
  • Consider performing some database normalization, at least 1NF or even 2NF/3NF
    • some of the suggestions above already lead into that direction).
    • not sure whether this would actually help us, though, considering we automatically create the content of the DB. However, since normalization helps keep data integretity, it might make incremental updating safe(er)
    • What else should be normalized? sections, maintainers, what else? (Chris01)

File cleanup

This refers to the PDB related files in our repository. Some of them could/should be removed, renamed, documented or otherwise modified.

The following files are in web/pdb:

• list.php, maintainer.php, nomaintainer.php, search.php, section.php: have all been changed to simply redirect to browse.php
• testing.php: can eventually be replaced by a redirect to browse.php (once we have refactored the "release" DB table appropriately)
• compare.php: allowed comparing trees, currently not functional: Remove it for now (can easily be restored from CVS should we need to reimplement this feature at some point)
• releases.inc: This should go, instead all the relevant information should be stored in the DB alone (this makes maintanance easier). For this, the "release" DB table needs to be refactored, see elsewhere on this page
• Suspected part of clef's work on .deb contents db DMacks
  • submit.php: Looks like dead code, remove it?
  • packagedetails.php: This seems to be "dead" code: It could in theory be reached by a hidden and currently not working "search package content" feature of search.php: Go to http://pdb.finkproject.org/pdb/search.php?c=1 . Could probably be completly removed

The following files are in scripts/pdb:

• HOWTO: Needs to be updated at some point. It seems that it hasn't received much attention in recent years, as it doesn't even mention pdb_full_update-piecewise and pdb_full_update-unified
• flag: This evil beast is used for setting the "latest" and "needtest" flags -- once we remove those, it can finally be gone for good, yay :)
• pdb_full_update-unified: This is long unused and no longer certain it's been updated in sync with -piecewise. No longer used because the full update can be too large or slow and hit CPU or timeout limits.
• pdb_full_update-piecewise: This is the actual update script run by cron to update the databases.

Frontend: Web interface

In this section, we are concerned with the user interface of the package database, i.e. the parts of it with which the user interacts.

Usability

• Limit the amount of data sets returned for large queries, could happen with a LIMIT in the DB, provided the whole result is cached
• Look into better display possibilities, like a faked scrolling that actually updates inside the page using AJAX
• Investigate meaningful colour coding (might also help visually impaired)
• Make sure there is a stripped down, no frills, no javascript version for command line access
• Move as much as possible of the layout into CSS
• Displayed advanced fields only when they are necessary in context or an advanced option is active
• Simplify, simplify and simplify
  • The two "Exclude..." options at the bottom of the "Browse..." search form seem redundant at first glance. Maybe this could be moved to another "Advanced" section. --Charleslepple 04:02, 13 March 2007 (CET)
• The 'Clear Form' button should always set the form fields back to default (emtpy, Any, etc.). This probably needs some javascripting. --chris01
• Improvements in parsing of Maintainer: when one puts in finchproject.org for the maintainer, all of the packages show up--similarly if I use akh@finkproject.org.--Alexkhansen 21:53, 12 March 2007 (CET)

Improved HTML code

• Add type="text/css" to every css link markups
• Copyright &copy 2001-2007 instead of Copyright copy 2001-2007
  • There was already an ampersand but the ending semicolon was missing in the lower-left of the webpages. Is that what you meant? DMacks 22:53, 26 January 2007 (CET)
• Encode the url, i.e : <a href="../maintainer.php?maintainer=Jeffrey%20Whitaker">
• I would have use a definition list dl, dt, dd for the part under the pkgversion table. Ex: http://www.maxdesign.com.au/presentation/definition/dl-table-display.htm
  • Seems reasonable. Also improves accessibility (logical relationship of dd to its dt vs just adjacent table cells). Can <div> still be used within dd so that we can manipulate the positioning of the full description?
    • Different span classes may be defined to handle the position of various parts of description. See an example here: http://micmacfr.homeunix.org/translate.shtml.shtml
• Display browse.php results in a table instead of a list? Might make it more readable.

PHP Code

• Add modularity to the code. I looked at browse.php and it is frightening. Everything is happening piece after piece without any logical workflow. I will try to rewrite it and will drop here a link for revision by others --Jimi Dini 17:49, 18 April 2007 (CEST)

Other

(possibly to be moved to another section)