the Fink Wiki - User contributions [en]

Fink:PackageDatabase:NewDesign

2007-09-24T13:15:41Z

Chris01: Howto is now updated /* File cleanup */

{{Fink_Header}}

== Ideas about Improving the PDB ==

This page collectes ideas on what could and should be improved in the [http://pdb.finkproject.org/pdb/ Package Database (pdb)].

== Testing Setup ==

A demo of how the new PDB could look can be found at http://finch.finkproject.org/~chris01/fink_web/pdb/ . The corresponding files are on finch.finkproject.org . The scripts can be found at:

<code>finch.finkproject.org:/home/chris01/new_pdb/scripts/pdb</code>

The files there are checked out from the [http://fink.cvs.sourceforge.net/fink/scripts/pdb/?pathrev=redesign_pdb redesign_pdb branch]. And the web pages are at:

<code>finch.finkproject.org:/home/chris01/public_html/fink_web/pdb</code>

also checked out from the [http://fink.cvs.sourceforge.net/fink/web/pdb/?pathrev=redesign_pdb redesign_pdb branch]. The [http://fink.cvs.sourceforge.net/fink/scripts/pdb/HOWTO?view=markup&pathrev=redesign_pdb HOWTO] file describes how to clean out the db and start fresh.

The database can be accessed through the [https://finch.finkproject.org/phpmyadmin/ phpMyAdmin web] interface.

'''IMPORTANT''': Always use the [http://fink.cvs.sourceforge.net/fink/scripts/pdb/update.sh?view=markup&pathrev=redesign_pdb update.sh] script to update the cvs checkouts. This scripts fixes the group permissions so that everybody has access to the files. However, for the php files there is no such script. So make sure you do:

<code>chmod -R g+w *</code>

== Backend: Implementation issues, Database Infrastructure ==

In this section, we talk about the "behind the scenes" code of the package database,
i.e. the parts not visible to users

=== General ===

* Incremental update of package db table. E.g.: ([[User:Chris01|chris01]])
** create a temporary table with new entries
** Remove everything from old table not in temp table
** Update entries already in old table
** Add entries only in temp table
** <s>and dmacks says: "Now just a minute, why doesn't index.php just do SELECT COUNT(DISTINCT name) FROM package; ?"</s> ([[User:Chris01|chris01]])
* <s>Links to the info files in cvs, and last changed date.</s> ([[User:Chris01|chris01]])
* <s>Better formatting of the Desc fields</s> ([[User:Chris01|chris01]])
* <s>Show info about a specific version, e.g. http://www.finkproject.org/pdb/package.php/cups-dev?version=1.1.15-2</s>([[User:Chris01|chris01]])
** <s>In the same vein, it should always be clearly visible which version is being displayed -- maybe display that information in the the title, so instead of "Package cups-dev" it shows "Package cups-dev 1.1.20-11"; alternatively, add a "Version:" entry to the table</s> ([[User:Fingolfin|Fingolfin]])
** <s>The table on package.php which shows the latest known versions of a given package for each dist/tree would probably be linking to these "specific version" pages</s> ([[User:Fingolfin|Fingolfin]])
* <s>exclude virtuals, or handle them gracefully, e.g. by showing all possible virtual packages without a version number, might be difficult</s> ([[User:Chris01|chris01]]: just don't show them.)
* <s>omit actual locally-installed fink if it is present</s>
* Why are there entries in the old pdb for packages that are no longer in any dist? (E.g. [http://pdb.finkproject.org/pdb/package.php/archive-zip-pm586-man archive-zip-pm586-man]) ([[User:Chris01|chris01]])
** They're available only in dists that aren't displayed in the table (flags/"latest" doesn't know releases.inc and package.php only looks at "latest"). archive-zip-pmXXX-man is present in current-10.4-transitional-unstable only. ([[User:Dmacks|DMacks]] 03:22, 30 January 2007 (CET))
** So we should discuss how we deal with entries of old releases. I suggest: don't show them in the index anymore. Objections? ([[User:Chris01|chris01]])
* <s>Show DescUsage</s> ([[User:Chris01|chris01]])
* Do we want to hide some information for the default package page, and have an expert mode (e.g. DescUsage, dependencies, CVS, etc.) ([[User:Chris01|chris01]])
* Show dependencies? ([[User:Chris01|chris01]]) And should that include BuildDepends? ([[User:Alexkhansen|akh]])
* <s>Fixed a bug where packages were shown in a bindist even though the license was 'Restrictive' at the time of the bindist (e.g. svn-ssl)</s> ([[User:Chris01|chris01]])
* How about displaying the output of "fink validate" for every .info file, too? (We would have to store it in the DB, too, for this to work, of course). Benefit: This might coerce some more packagers into validating their packages, as they get a nice shiny green "Package is valid" vs. a list of yellow/orange "Validation warnings/errors". ([[User:Fingolfin|Fingolfin]])
* Should 'htdocs/pdb/db.inc.php' used in header.inc be the same as 'htdocs/db.inc.php'? ([[User:Chris01|chris01]])
* <s>Improve selecting splitoffs in package.php (include version info in link) ([[User:Chris01|chris01]])</s>
* <s>Make links to splitoffs version/release specific, if package.php was called with version/release parameters ([[User:Chris01|chris01]])</s>
* Is "[http://sourceforge.net/tracker/?func=detail&aid=1606084&group_id=17203&atid=117203 pdb doesn't see the "current" distribution (Fink Bug # 1606084)]" solved?

=== Changes to the DB table layout ===

* The PDB SQL format is not in the proper form for a relational DB ([[User:Fingolfin|Fingolfin]])
** <s>fullname is redundant and should go</s>
** <s>latest and needtest appear redundant, too</s> ([[User:Chris01|Chris01]])
** <s>desclong is preformated, instead of leaving this formatting to the presentation layer</s>
** <s>"release" table should be refactored:</s>
*** <s>the name "release" for the table / "package" field seems inappropriate. How about changing it to "distribution" ?</s>
*** <s>or rather, it might be split (for the sake of normalization): have a distribution" table, which contains: OS/distro, and priority</s>
*** <s>then, have a "release" (better name?) table, which contains: distro, type (stable/unstable/bindist), version (like "0.8.1" or "current").</s>
*** <s>that way, we have normalized the data, and get rid of releases.inc as it is now trivial to find the stable tree matching an unstable tree and vice versa.</s>
*** <s>however, I am not quite sure how one would now properly "link" packages to their release? Either we keep the current "release name" (somewhat contrary to normalization), or we simply number the releases sequentially, and use that number as key).</s>
*** <s>I suggest the the following DB schema and content for the dist/release tables: [[Fink:PackageDatabase:NewSchema]] ([[User:Chris01|Chris01]])</s>
** Consider performing some [http://en.wikipedia.org/wiki/Database_normalization database normalization], at least 1NF or even 2NF/3NF
*** some of the suggestions above already lead into that direction).
*** not sure whether this would actually help us, though, considering we automatically create the content of the DB. However, since normalization helps keep data integretity, it might make incremental updating safe(er)
*** What else should be normalized? sections, maintainers, what else? ([[User:Chris01|Chris01]])

=== File cleanup ===
This refers to the PDB related files in our repository. Some of them could/should be removed, renamed, documented or otherwise modified.

The following files are in web/pdb:
* <s>list.php, maintainer.php, nomaintainer.php, search.php, section.php: have all been changed to simply redirect to browse.php</s>
* <s>testing.php: can eventually be replaced by a redirect to browse.php (once we have refactored the "release" DB table appropriately)</s>
* <s>compare.php: allowed comparing trees, currently not functional: Remove it for now (can easily be restored from CVS should we need to reimplement this feature at some point)</s>
* <s>releases.inc: This should go, instead all the relevant information should be stored in the DB alone (this makes maintanance easier). For this, the "release" DB table needs to be refactored, see elsewhere on this page</s>
* <s>Suspected part of clef's work on .deb contents db [[User:Dmacks|DMacks]]</s>
** <s>submit.php: Looks like dead code, remove it?</s>
** <s>packagedetails.php: This seems to be "dead" code: It could in theory be reached by a hidden and currently not working "search package content" feature of search.php: Go to http://pdb.finkproject.org/pdb/search.php?c=1 . Could probably be completly removed</s>

The following files are in scripts/pdb:
* <s>HOWTO: Needs to be updated at some point. It seems that it hasn't received much attention in recent years, as it doesn't even mention pdb_full_update-piecewise and pdb_full_update-unified</s>
* <s>flag: This evil beast is used for setting the "latest" and "needtest" flags -- once we remove those, it can finally be gone for good, yay :)</s>
* <s>pdb_full_update-unified: This is long unused and no longer certain it's been updated in sync with -piecewise. No longer used because the full update can be too large or slow and hit CPU or timeout limits.</s>
* <s>pdb_full_update-piecewise: This is the actual update script run by cron to update the databases.</s>

== Frontend: Web interface ==

In this section, we are concerned with the user interface of the package database,
i.e. the parts of it with which the user interacts.

=== Usability ===

* Limit the amount of data sets returned for large queries, could happen with a LIMIT in the DB, provided the whole result is cached
* Look into better display possibilities, like a faked scrolling that actually updates inside the page using AJAX
* Investigate meaningful colour coding (might also help visually impaired)
* Make sure there is a stripped down, no frills, no javascript version for command line access
* Move as much as possible of the layout into CSS
* <s>Displayed advanced fields only when they are necessary in context or an advanced option is active</s>
* <s>Simplify, simplify and simplify</s>
** <s>The two "Exclude..." options at the bottom of the "Browse..." search form seem redundant at first glance. Maybe this could be moved to another "Advanced" section. --[[User:Charleslepple|Charleslepple]] 04:02, 13 March 2007 (CET)</s>
* <s>The 'Clear Form' button should always set the form fields back to default (emtpy, Any, etc.). This probably needs some javascripting. --[[User:chris01|chris01]]</s>
*<s>Improvements in parsing of Maintainer: when one puts in ''finchproject.org'' for the maintainer, [[all]] of the packages show up--similarly if I use ''akh@finkproject.org''.</s>--[[User:Alexkhansen|Alexkhansen]] 21:53, 12 March 2007 (CET)

=== Improved HTML code ===

*<s>Add type="text/css" to every css link markups</s>
*<s>Copyright &copy 2001-2007 instead of Copyright copy 2001-2007</s>
** <s>There was already an ampersand but the ending semicolon was missing in the lower-left of the webpages. Is that what you meant?</s> [[User:Dmacks|DMacks]] 22:53, 26 January 2007 (CET)
* <s>Encode the url, i.e : <a href="../maintainer.php?maintainer=Jeffrey%20Whitaker"></s>
*I would have use a definition list dl, dt, dd for the part under the pkgversion table. Ex: http://www.maxdesign.com.au/presentation/definition/dl-table-display.htm
**Seems reasonable. Also improves accessibility (logical relationship of dd to its dt vs just adjacent table cells). Can <div> still be used within dd so that we can manipulate the positioning of the full description?
***Different span classes may be defined to handle the position of various parts of description. See an example here: http://micmacfr.homeunix.org/translate.shtml.shtml
* <s>Display browse.php results in a table instead of a list? Might make it more readable.</s>

=== PHP Code ===

*Add modularity to the code. I looked at browse.php and it is frightening. Everything is happening piece after piece without any logical workflow. I will try to rewrite it and will drop here a link for revision by others --[[User:Jimi Dini|Jimi Dini]] 17:49, 18 April 2007 (CEST)

=== Other ===
(possibly to be moved to another section)

{{Fink_Header}}

[[Category: Fink|Package Database New Redesign]]
[[Category: Fink Structure|Package Database New Redesign]]

Fink:PackageDatabase:NewSchema

2007-02-20T19:49:52Z

Chris01: collate ascii_general_ci

{{Fink_Header}}

Possible new layout for a distribution/release db. It should replace the existing release table and also the [http://fink.cvs.sourceforge.net/fink/web/pdb/releases.inc?view=markup&pathrev=redesign_pdb releases.inc] file. The code below will go into the [http://fink.cvs.sourceforge.net/fink/scripts/pdb/schema.sql?view=markup&pathrev=redesign_pdb schema.sql] and [http://fink.cvs.sourceforge.net/fink/scripts/pdb/releases.sql?view=markup&pathrev=redesign_pdb releases.sql]

<pre>
create table if not exists distribution (
dist_id int unsigned not null auto_increment,
identifier varchar(16) not null,
description varchar(64) not null default '',
architecture enum('powerpc', 'i386') not null default 'powerpc',
priority tinyint unsigned not null default '1',
active boolean default 1,
primary key (dist_id)
) collate ascii_general_ci;

create table if not exists distribution_releases (
rel_id int unsigned not null auto_increment,
dist_id int unsigned not null references distribution(dist_id),
type enum('bindist', 'stable', 'unstable') not null,
version varchar(16) not null,
priority tinyint unsigned not null default '1',
active boolean default 1,
primary key (rel_id)
) collate ascii_general_ci;

insert into distribution (dist_id, identifier, description, architecture, priority, active) values (null, '10.1', '10.1', 'powerpc', 1, 1);
select @last_dist_id := last_insert_id();
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'bindist', '0.4.1', 1, 1);

insert into distribution (dist_id, identifier, description, architecture, priority, active) values (null, '10.2-gcc3.3', '10.2\n(gcc3.3 only)', 'powerpc', 2, 1);
select @last_dist_id := last_insert_id();
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'unstable', 'current', 3, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'stable', 'current', 2, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'bindist', '0.6.4', 1, 1);

insert into distribution (dist_id, identifier, description, architecture, priority, active) values (null, '10.3', '10.3', 'powerpc', 3, 1);
select @last_dist_id := last_insert_id();
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'unstable', 'current', 3, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'stable', 'current', 2, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'bindist', '0.7.2', 1, 1);

insert into distribution (dist_id, identifier, description, architecture, priority, active) values (null, '10.4', '10.4/powerpc', 'powerpc', 4, 1);
select @last_dist_id := last_insert_id();
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'unstable', 'current', 3, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'stable', 'current', 2, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'bindist', '0.8.1', 1, 1);

insert into distribution (dist_id, identifier, description, architecture, priority, active) values (null, '10.4', '10.4/intel', 'i386', 5, 1);
select @last_dist_id := last_insert_id();
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'unstable', 'current', 3, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'stable', 'current', 2, 1);
insert into distribution_releases (rel_id, dist_id, type, version, priority, active) values (null, @last_dist_id, 'bindist', '0.8.1', 1, 1);
</pre>

And the corresponding package schema:
<pre>
create table if not exists package (
pkg_id int unsigned not null auto_increment,
rel_id int unsigned not null references distribution_releases(rel_id),
name varchar(64) not null,
parentname varchar(64),
version varchar(64) not null,
revision varchar (16) not null,
epoch tinyint not null default '0',
descshort varchar(80) not null default '',
desclong text,
descusage text,
maintainer varchar(255),
license varchar(64),
homepage varchar(255),
section varchar(32) not null,
infofile varchar(255) not null default '',
infofilechanged datetime,
primary key (pkg_id),
index (name),
index (section),
index (rel_id)
) collate ascii_general_ci;
</pre>

{{Fink_Header}}

[[Category: Fink|Package Database New Schema]]
[[Category: Fink Structure|Package Database New Schema]]

Fink:Policy:Packaging Java

2007-02-20T14:44:00Z

Chris01: Reverted edit of Zenek, changed back to last version by RangerRick

{{Fink_Header}}

There is no official "policy" per se for packaging Java software in Fink, but there are a number of best practices that make it easier.

== The '''''Type''''' Field ==

By default, if you use <code>Type: java</code> without specifying a version, Fink will use the '''lowest''' JDK found in /System/Library/Frameworks/JavaVM.framework/Versions. So if you have a 1.3.1, 1.4.2, and 1.5.0 JDK on your system, it will use the 1.3.1.

If you use <code>Type: java('''version''')</code> (i.e., <code>Type: java(1.4)</code>), it will use the '''highest''' JDK found that matches that version. So if you have both a 1.4.1 and 1.4.2 JDK on your system, it will use the 1.4.2 JDK.

Regardless of whether you use the versioned or unversioned field, as long as you're setting <code>Type: java</code>, the build environment inside your package will have the following extra environment variables set:

JAVA_HOME /System/Library/Frameworks/JavaVM.framework/Versions/''<code>detected_version</code>''/Home
PATH /System/Library/Frameworks/JavaVM.framework/Versions/''<code>detected_version</code>''/Home/bin:$PATH

== Using the '''''Type''''' Field Effectively ==

There are 2 major ways it is recommended to package Java software, depending on whether you want to require a specific JDK, or you want to target a specific JDK or higher.

=== Requiring a Specific JDK ===

If you want to ensure your software is compiled with, say, the 1.4 JDK, use:

Type: java(1.4)

This will detect a JDK of that specific major and minor version number, and set <code>JAVA_HOME</code> and <code>PATH</code> respectively. If the software you are packaging uses <code>javac -target</code> (or ant's <javac target=""/> notation), it will still use the JDK that you specified, but generate bytecode compatible with the target given.

=== Using The Available JDK (recommended) ===

The problem with using <code>Type: java(1.4)</code> is that it requires a specific JDK, even if later ones are available. The primary reason to do so is to be able to create bytecode usable by older JDKs.

Instead, it is recommended to do the following:

Package: myjavapackage
Type: java
Depends: system-java (>= 1.4-1)
BuildDepends: system-java-dev (>= 1.4-1)
Patch: %n.patch

Create a patch that makes sure that javac is being called with a target of 1.4. In the case of an ant build.xml, change any entries like:

<javac srcdir="foo">
<patternset refid="foo.files" />
</javac>

...into...

<javac target="1.4" srcdir="foo">
<patternset refid="foo.files" />
</javac>

In the case of some other tool calling <code>javac</code>, you can pass "<code>-target</code>" on the <code>javac</code> command-line.

This lets you not care what JDK is installed, as long as it's 1.4 or newer.

== Problems with specific JDKs ==

; as of release 1.5, 'enum' is a keyword, and may not be used as an identifier : The 1.5 JDK adds some new keywords to the language, like <code>enum</code>. In some cases, you might need to add <code>source="1.4"</code> to your build.xml (or <code>-source 1.4</code> on the javac command-line) to make things work correctly.

{{Fink_Header}}

[[Category: Fink|Policy:Packaging Java]]
[[Category: Fink_Documentation|Policy:Packaging Java]]

Fink:Roadmap:0.25 new features

2006-12-12T21:03:46Z

Chris01: changes based on dmacks comments

{{Fink_Header}}

===New features in fink 0.25===

* [[Fink:Roadmap:0.25 new features:Incremental indexing|Incremental Indexing]]

* SkipPrompts
** Any more categories users would like?

* Info3
** Can indent nicely in .info files
** No more support for RFC-822 multi-lines
** Can put comments in pkglist fields

* fast scanpackages with apt-ftparchive

* auto-detection of country

* Validator fixes:
** detect InfoN property (should be wrapper)

* New <code>--trees</code> option restricts fink to using .info files in the chosen tree(s).

* new [[Fink:fink cleanup|fink cleanup]] syntax

* New <code>--maintainer</code> mode

* New <code>--log-output</code> and <code>--logfile</code> flags to save transcripts of package building process

* Integrated scanpackages, including ability to not index restrictive packages.

* [[Fink:Major_New_Feature_Plans:Test_suite|Support for running package test suites]]

* reevaluate fink --tree=stable list
* Validator:
** Complain about [https://sourceforge.net/tracker/index.php?func=detail&aid=702802&group_id=17203&atid=367203 > and < in dependency versioning] (in .info).
** dmacks: complete overhaul of .deb (val-unpack branch)
** InfoN validator patch '''vasi fixed it the way he likes it'''
* RangerRick: 'fink build foo-1.0-1' where a .deb exists in the bindist for foo-1.0-1 and UseBinaryDist is true should build locally, not fetch from the bindist.
* Info3: allow comment lines in pkglist fields
<pre>Depends: <<
#foo-shlibs, foo is static-only right now
bar-shlibs
<<</pre>
* <code>chown -h</code> functionality during --build-as-nobody (use Command:chowname_hr?)
* dmacks: overhaul buildlocks to go away more automatically and have a cleaner cleanup
** Automatically do cleanup when embarking on a package installation or removal
* dmacks: buildlock destruction
** Checking via lock_wait and PreRm
** Removal on build failure via Fink::Finally
* remove line-number in SysState error
* BuildConflicts breakage
* chown_hR is still leaving nobody-owned links.
* progress bar during indexing ("." every 100 .info processed or every 10% of set of .info processed?)—newbies to HEAD have complained <code>index -f</code> "has hung"...can take up to 5 minutes
* alternate checksum algorithms

===Bug fixes in fink 0.25===

* Fink no longer gets confused by debs in a local apt repository

* When asking to choose mirrors, fink orders choices alphabetically properly, and presents a reasonable set of mirrors when the user asks for all those on a continent.

* BuildConflicts should actually work

* many more we've forgotten all about...

{{Fink_Header}}

[[Category: Fink|Roadmap 0.25 New Features]]
[[Category: Fink_Roadmap|0.25 New Features]]