

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.5 - (show annotations) (download)
Thu Sep 15 21:02:11 2005 UTC (9 years, 3 months ago) by g2boojum
Branch: MAIN
Changes since 1.4: +8 -4 lines
File MIME type: text/plain

1 GLEP: 33
2 Title: Eclass Restructure/Redesign
3 Version: $Revision: 1.4 $
4 Last-Modified: $Date: 2005/09/15 02:37:38 $
6 Status: Approved
7 Type: Standards Track
8 Content-Type: text/x-rst
9 Created: 29-Jan-2005
12 Status
13 ======
15 Approved by the Gentoo Council on 15 September 2005.
17 Abstract
18 ========
20 For any design, the transition from theoretical to applied exposes inadequacies
21 in the original design. This document is intended to document, and propose a
22 revision of the current eclass setup to address current eclass inadequacies.
24 This document proposes several things- the creation of ebuild libraries, 'elibs',
26 addition of changelogs, and a way to allow for simple eclass gpg signing.
27 In general, a large scale restructuring of what eclasses are and how they're
28 implemented. Essentially version two of the eclass setup.
31 Terminology
32 ===========
36 distinction is elaborated on within this document.
39 Motivation and Rationale
42 Eclasses within the tree currently are a bit of a mess- they're forced to
43 maintain backwards compatibility w/ all previous functionality. In effect,
44 their api is constant, and can only be added to- never changing the existing
45 functionality. This obviously is quite limiting, and leads to cruft accruing in
46 eclasses as a eclasses design is refined. This needs to be dealt with prior to
47 eclass code reaching a critical mass where they become unmanageable/fragile
48 (recent pushes for eclass versioning could be interpreted as proof of this).
50 Beyond that, eclasses were originally intended as a method to allow for ebuilds
53 the current design. Eclasses inherit other eclasses to get a single function- in
54 doing so, modifying the the exported 'template' (default src_compile, default
56 function, not making their eclass sensitive to changes in the template of the
57 eclass it's inheriting. The eclass designer -should- be aware of changes in the
58 function they're using, but shouldn't have to worry about their default src_*
59 and pkg_* functions being overwritten, let alone the env changes.
61 Addressing up front why a collection of eclass refinements are being rolled into
63 phases. Why do it though? It's simpler for developers to know that the first
64 eclass specification was this, and that the second specification is that,
65 rather then requiring them to be aware of what phase of eclass changes is in
66 progress.
68 By rolling all changes into one large change, a line is intentionally drawn in
69 the sand. Old eclasses allowed for this, behaved this way. New eclasses allow
70 for that, and behave this way. This should reduce misconceptions about what is
71 allowed/possible with eclasses, thus reducing bugs that result from said
72 misconceptions.
74 A few words on elibs- think of them as a clear definition between behavioral
75 functionality of an eclass, and the library functionality. Eclass's modify
77 common bash functionality.
79 Consider the majority of the portage bin/* scripts- these all are candidates for
80 being added to the tree as elibs, as is the bulk of eutils.
83 Specification
86 The various parts of this proposal are broken down into a set of changes and
87 elaborations on why a proposed change is preferable. It's advisable to the
88 reader that this be read serially, rather then jumping around.
91 Ebuild Libraries (elibs for short)
94 As briefly touched upon in Motivation and Rationale, the original eclass design
95 allowed for the eclass to modify the metadata of an ebuild, metadata being the
97 and used by portage for dep resolution, fetching, etc. Using the earlier
98 example, if you're after a single function from an eclass (say epatch from
99 eutils), you -don't- want the metadata modifications the eclass you're
100 inheriting might do. You want to treat the eclass you're pulling from as a
101 library, pure and simple.
104 as a repository of ebuild function libraries. Rather then relying on using the
105 source command, an 'elib' function should be added to portage to import that
106 libraries functionality. The reason for the indirection via the function is
107 mostly related to portage internals, but it does serve as an abstraction such
108 that (for example) zsh compatibility hacks could be hidden in the elib function.
110 Elib's will be collections of bash functions- they're not allowed to do anything
111 in the global scope aside from function definition, and any -minimal-
112 initialization of the library that is absolutely needed. Additionally, they
113 cannot modify any ebuild template functions- src_compile, src_unpack. Since they are
114 required to not modify the metadata keys, nor in any way affect the ebuild aside
115 from providing functionality, they can be conditionally pulled in. They also
117 other elibs. A real world example would be the eutils eclass.
119 Portage, since the elib's don't modify metadata, isn't required to track elibs
121 forced to be regenerated/marked stale when changed (this is more of an infra
122 benefit, although regen's that take too long due to eclass changes have been
123 known to cause rsync issues due to missing timestamps).
129 completely incapable of modifying metadata. There is no room for confusion, late loading
133 and is an ancillary benefit of the first reason.
143 multiple times, as eclasses are.
145 To clarify, for example this is invalid.
146 ::
148 if [[ -n ${SOME_VAR} ]]; then
150 else
152 fi
155 Regarding maintainability of elibs, it should be a less of a load then old
156 eclasses. One of the major issues with old eclasses is that their functions are
157 quite incestuous- they're bound tightly to the env they're defined in. This
158 makes eclass functions a bit fragile- the restrictions on what can, and cannot
159 be done in elibs will address this, making functionality less fragile (thus a
160 bit more maintainable).
162 There is no need for backwards compatibility with elibs- they just must work
163 against the current tree. Thus elibs can be removed when the tree no longer
164 needs them. The reasons for this are explained below.
166 Structuring of the elibs directory will be exactly the same as that of the new
167 eclass directory (detailed below), sans a different extension.
172 such that no misconceptions occur, resulting in bugs.
174 The reduced role of Eclasses, and a clarification of existing Eclass requirements
177 Since elibs are now intended on holding common bash functionality, the focus of
178 eclasses should be in defining an appropriate template for ebuilds. For example,
179 defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc.
180 Additionally, eclasses should pull in any elibs they need for functionality.
182 Eclass functionality that isn't directly related to the metadata, or src_* and
183 pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This
184 however isn't a hard requirement, merely a strongly worded suggestion.
186 Previously, it was 'strongly' suggested by developers to avoid having any code
187 executed in the global scope that wasn't required. This suggestion is now a
188 requirement. Execute only what must be executed in the global scope. Any code
189 executed in the global scope that is related to configuring/building the package
191 an absolute requirement to clarify it) *must* be constant. The results of
193 the keys exported on system B.
195 If an eclass (or ebuild for that matter) violates this constant requirement, it
196 leads to portage doing the wrong thing for rsync users- for example, wrong deps
197 pulled in, leading to compilation failure, or dud deps.
199 If the existing metadata isn't flexible enough for what is required for a
200 package, the parsing of the metadata is changed to address that. Cases where
201 the constant requirement is violated are known, and a select few are allowed-
202 these are exceptions to the rule that are required due to inadequacies in
203 portage. Any case where it's determined the constant requirement may need to be
205 devs. This should be done prior to committing.
209 failures and unneeded deps being pulled in.
211 After that stern reminder, back to new eclasses. Defining INHERITED and ECLASS
212 within the eclass is no longer required. Portage already handles those vars if
213 they aren't defined.
215 As with elibs, it's no longer required that backwards compatibility be maintained
216 indefinitely- compatibility must be maintained against the current tree, but
217 just that. As such new eclasses (the true distinction of new vs old is
218 elaborated in the next section) can be removed from the tree once they're no
219 longer in use.
222 The end of backwards compatibility...
227 have *ancient* eclasses that are completely unused sitting in the tree, for
228 example inherit.eclass. The reason for this, not surprisingly, is a portage
229 deficiency: on unmerging an installed ebuild, portage used the eclass from the
230 current tree.
234 older glibc version during an upgrade to a newer version. So either the glibc
235 maintainer is left with the option of leaving people using ancient versions out
236 in the rain, or maintaining an ever increasing load of backwards compatibility
237 cruft in any used eclasses.
239 Binpkgs suffer a similar fate. Merging of a binpkg pulls needed eclasses from
243 The next major release of portage will address this- the environment that the
244 ebuild was built in already contains the eclasses functions, as such the env can
246 installed ebuilds will no longer go and pull needed eclasses from the tree,
249 So the backwards compatibility requirement for users of the next major portage
250 version (and beyond) isn't required. All the cruft can be dropped.
252 The problem is that there will be users using older versions of portage that don't
253 support this functionality- these older installations *cannot* use the
254 new eclasses, due to the fact that their portage version is incapable of
256 result in user-visible failures during unmerging.
259 a means to basically disallow access to the new eclasses for all portage versions
260 incapable of properly handling the env requirements.
262 Unfortunately, we cannot just rely on a different grouping/naming convention within
263 the old eclass directory. The new eclasses must be inaccessible, and portage throws
264 a snag into this- the existing inherit function that is used to handle existing
265 eclasses. Basically, whatever it's passed (inherit kernel or inherit
266 kernel/kernel) it will pull in (kernel.eclass, and kernel/kernel.eclass
267 respectively). So even if the new eclasses were implemented within a
268 subdirectory of the eclass dir in the tree, all current portage versions would
269 still be able to access them.
271 In other words, these new eclasses would in effect, be old eclasses since older
272 portage versions could still access them.
275 Tree restructuring
278 There are only two way to block the existing (as of this writing) inherit
279 functionality from accessing the new eclasses- either change the extension of
281 subdirectory of the tree then eclass.
283 The latter is preferable, and the proposed solution. Reasons are- the current
284 eclass directory is already overgrown. Structuring of the new eclass dir
285 (clarified below) will allow for easier signing, ChangeLogs, and grouping of
286 eclasses. New eclasses allow for something akin to a clean break and have new
287 capabilities/requirements, thus it's advisable to start with a clean directory,
288 devoid of all cruft from the old eclass implementation.
290 If it's unclear as to why the old inherit function *cannot* access the new
291 eclasses, please reread the previous section. It's unfortunately a requirement
292 to take advantage of all that the next major portage release will allow.
297 possibility (likely a fact) that new eclasses someday may be found lacking, and
299 functionality with direct relation to sourcing new files, and we would then need
300 to further populate ${PORTDIR}.
304 ::
305 kernel/
306 kernel/linux-info.eclass
307 kernel/linux-mod.eclass
308 kernel/kernel-2.6.eclass
309 kernel/kernel-2.4.eclass
310 kernel/ChangeLog
311 kernel/Manifest
313 No eclasses will be allowed in the base directory- grouping of new eclasses will
314 be required to help keep things tidy, and for the following reasons. Grouping
315 of eclasses allows for the addition of ChangeLogs that are specific to that
316 group of eclasses, grouping of files/patches as needed, and allows for
317 saner/easier signing of eclasses- you can just stick a signed
318 Manifest file w/in that grouping, thus providing the information portage needs
319 to ensure no files are missing, and that nothing has been tainted.
321 The elib directory will be structured in the same way, for the same reasons.
323 Repoman will have to be extended to work within new eclass and elib groups, and
324 to handle signing and committing. This is intentional, and a good thing. This
325 gives repoman the possibility of doing sanity checks on elibs/new eclasses.
327 Note these checks will not prevent developers from doing dumb things with eclass-
329 There is no way to prevent people from doing dumb things (exempting perhaps repeated
330 applications of a cattle prod)- these are strictly automatic checks, akin to repoman's
331 dependency checks.
334 The start of a different phase of backwards compatibility
337 As clarified above, new eclasses will exist in a separate directory that will be
341 this for rsync users.
343 There still is the issue of users who haven't upgraded to the required portage
344 version. This is a minor concern frankly- portage releases include new
345 functionality, and bug fixes. If they won't upgrade, it's assumed they have
346 their reasons and are big boys, thus able to handle the complications themselves.
348 The real issue is broken envs, whether in binpkgs, or for installed packages.
349 Two options exist- either the old eclasses are left in the tree indefinitely, or
351 can be merged.
353 Shifting them out of the tree is advisable for several reasons- less cruft in
355 for attack). Note that the proposed method of eclass signing doesn't even try
356 to address them. Frankly, it's not worth the effort supporting two variations
357 of eclass signing, when the old eclass setup isn't designed to allow for easy
358 signing.
360 If this approach is taken, then either the old eclasses would have to be merged
361 to an overlay directory's eclass directory (ugly), or to a safe location that
362 portage's inherit function knows to look for (less ugly).
364 For users who do not upgrade within the window of N months while the old
366 If they specifically block the new portage version, as the ebuilds in the tree
367 migrate to the new eclasses, they will have less and less ebuilds available to
368 them. If they tried injecting the new portage version (lying to portage,
369 essentially), portage would bail since it cannot find the new eclass.
370 For ebuilds that use the new eclasses, there really isn't any way to sidestep
371 the portage version requirement- same as it has been for other portage features.
374 if a user has not upgraded to a portage version supporting env processing, they
375 will lose the ability to unmerge any installed ebuild that used an old
376 eclass. Same cause, different symptom being they will lose the ability to merge
377 any tbz2 that uses old eclasses also.
380 has suffered significant corruption of their installed package database (vdb). This is
383 In such a case, even with the new portage capabilities, they would need
384 the old eclass compat ebuild.
392 truly the least of the users concern in such a case.
394 Continuing with the more likely scenario, users unwilling to upgrade portage will
396 the missing eclasses, thus providing that lost functionality.
398 Note the intention isn't to force them to upgrade, hence the ability to restore the
399 lost functionality. The intention is to clean up the existing mess, and allow us
401 is akin, exempting the fact we're providing a way to make the eggs whole again
402 (the king's men would've loved such an option).
405 Migrating to the new setup
408 As has been done in the past whenever a change in the tree results in ebuilds
409 requiring a specific version of portage, as ebuilds migrate to the new eclasses,
410 they should depend on a version of portage that supports it. From the users
411 viewpoint, this transparently handles the migration.
413 This isn't so transparent for devs or a particular infrastructure server however.
414 Devs, due to them using cvs for their tree, lack the pregenerated cache rsync
415 users have. Devs will have to be early adopters of the new portage. Older
416 portage versions won't be able to access the new eclasses, thus the local cache
417 generation for that ebuild will fail, ergo the depends on a newer portage
418 version won't transparently handle it for them.
420 Additionally, prior to any ebuilds in the tree using the new eclasses, the
421 infrastructure server that generates the cache for rsync users will have to
422 either be upgraded to a version of portage supporting new eclasses, or patched.
423 The former being much more preferable then the latter for the portage devs.
425 Beyond that, an appropriate window for old eclasses to exist in the tree must be
427 so users can get the old eclasses if needed.
430 transfer the old eclass into an appropriate grouping in the new eclass directory,
431 although it's advisable they cleanse all cruft out of the eclass. You can
432 migrate ebuilds gradually over to the new eclass, and don't have to worry about
433 having to support ebuilds from X years back.
436 window in which to redesign it. It's humbly suggested eclass devs take
437 advantage of it. :)
440 Backwards Compatibility
443 All backwards compatibility issues are addressed in line, but a recap is offered-
444 it's suggested that if the a particular compatibility issue is
445 questioned/worried over, the reader read the relevant section. There should be
446 a more in depth discussion of the issue, along with a more extensive explanation
447 of the potential solutions, and reasons for the chosen solution.
449 To recap:
450 ::
452 New eclasses and elib functionality will be tied to a specific portage
453 version. A DEPENDs on said portage version should address this for rsync
454 users who refuse to upgrade to a portage version that supports the new
455 eclasses/elibs and will gradually be unable to merge ebuilds that use said
456 functionality. It is their choice to upgrade, as such, the gradual
457 'thinning' of available ebuilds should they block the portage upgrade is
458 their responsibility.
460 Old eclasses at some point in the future should be removed from the tree,
461 and released in a tarball/ebuild. This will cause installed ebuilds that
462 rely on the old eclass to be unable to unmerge, with the same applying for
463 merging of binpkgs dependent on the following paragraph.
465 The old eclass-compat is only required for users who do not upgrade their
466 portage installation, and one further exemption- if the user has somehow
468 in the process, they've lost their saved environments. The eclass-compat
469 ebuild would be required for ebuilds that required older eclasses in such a
470 case. Note, this case is rare also- as clarified above, it's mentioned
471 strictly to be complete, it's not much of a real world scenario as elaborated
472 above.
475 Copyright
476 =========
478 This document has been placed in the public domain.

  ViewVC Help
Powered by ViewVC 1.1.20