A PDF version of this document is also available (PDF version, 440KB).
Timothy Arnold-Moore, Legislation Management Consultant
InQuirion Pty Ltd
The TeraText® trademark and logo are owned by Scientific Applications International Corporation.
Table of Contents
1. Executive summary
1.1 Purpose
1.2 The PAL project
1.3 Stakeholders
1.4 Process
1.5 Summary of recommendations
1.5.1 Reassess rendering engine choice
1.5.2 Consider updating to Documentum 5
1.5.3 Upgrade screens
1.5.4 Use native Unicode for macrons throughout the system
1.5.5 Consolidation of the Authoring Tool documentation
1.5.6 Make user acceptance testing team available throughout development
1.5.7 Consider more redundancy for the systems critical for publication
1.5.8 Consider upgrading the web engine to include an XML repository capability
1.6 Summary
2. Introduction
2.1 Background
2.2 A construction project analogy
2.3 Why legislation is different
3. Stakeholders
3.1 People of New Zealand
3.2 The Government of New Zealand
3.3 Parliament
3.4 Parliamentary Counsel Office (PCO)
3.4.1 PCO Drafters
3.4.2 Secretaries and support staff
3.4.3 Proofreaders
3.4.4 PPU
3.4.5 Compilers
3.4.6 IT Team
3.5 Inland Revenue Department (IRD) Drafters
3.6 Office of the Clerk (OC)
3.7 SecuraCopy
3.8 Datacom
3.9 Parliamentary Service
3.10 Legal publishers
3.11 Unisys
3.12 Summary
4. Process
4.1 Initial review of system documentation
4.2 On-site visit
4.2.1 Focus groups and user interviews
4.2.2 System demonstration
4.2.3 System architecture briefing
4.2.4 Code reviews and walk-throughs
4.2.5 Summary of the on-site schedule
4.3 Vendor directions
4.4 Preparation of draft report
4.5 Presentation of draft report
5. Architecture
5.1 Typical architecture for legislative drafting
5.1.1 Authoring Tool
5.1.2 Repository
5.1.3 Workflow
5.1.4 Consolidation tool
5.1.5 Print delivery
5.1.6 Electronic delivery
5.2 The PAL architecture
5.2.1 Authoring/Consolidation tool
5.2.2 Repository
5.2.3 Workflow
5.2.4 Print delivery
5.2.5 Electronic delivery
5.3 PAL development environments
5.3.1 FOSI
5.3.2 ACL (ArborText)
5.3.3 DocBasic (Documentum)
5.3.4 Java
5.3.5 XSLT
5.3.6 CSS
5.3.7 C# and .NET
5.3.8 Microsoft Word
5.4 Conclusions about the PAL architecture
6. Specific questions
6.1 Is the application architecture implemented in a consistent, logical and understandable manner?
6.2 Does the Epic software (Epic Editor, Print Composer, and E3) have the capability to meet PCO's requirements?
6.2.1 Epic Editor—the Authoring Tool
6.2.2 Epic Print Composer—the Print Rendering Tool
6.3 Does Documentum 4i have the capability to meet PCO's requirements?
6.4 Does the mix of package customisation and bespoke development support future development and package upgrade without major rewrites or design changes?
6.4.1 Authoring Tool
6.4.2 Rendering Engine
6.4.3 CMS
6.4.4 Website
6.5 Are recognisable standards applied and consistently implemented?
6.5.1 XML
6.5.2 FOSI
6.5.3 XSLT
6.5.4 Java
6.5.5 .NET
6.6 Is the use of a coding standard apparent, and has it been universally and consistently applied?
6.7 Are APIs well defined and documented?
6.8 Is sufficient information available in terms of documentation and coding structure for a developer or designer to be able to make modifications without the need to rewrite components?
6.9 Given the manner in which XML technology has been implemented, are there any implications in terms of technology development, maintenance and upgrade of components, and data portability?
6.10 Are there any demonstrable gaps or deficiencies in the system documentation and system test plans?
6.10.1 System documentation
6.10.2 System test plans
7. Other issues
7.1 Ongoing support, maintenance and development
7.1.1 Business requirements
7.1.2 Context
7.1.3 Skill sets
7.1.4 Options
7.1.5 Risks
7.2 Print rendering test set
7.3 Legacy data
7.4 Security issues
7.5 Uptime issues
7.5.1 Website
7.5.2 VPN
7.6 Compilation and reprints
7.7 Cross referencing tool
7.8 Annual volumes
7.9 Naming of transform output files
1. Executive summary
InQuirion has undertaken a technical review of the Public Access to Legislation (PAL) project for the New Zealand Parliamentary Counsel Office (PCO) to enable PCO to reassure the New Zealand Government, as sponsors of the PAL project, that the PAL system, when implemented, will be operationally stable, maintainable, and capable of supporting future enhancement and development. This document captures the results of that review.
The PAL project is a large information technology project designed to facilitate the drafting, management, and delivery of legislation by the Parliamentary Counsel Office (PCO), Office of the Clerk (OC), Tax Drafting Unit of Inland Revenue Department (IRD), and external contractors Brookers Ltd, SecuraCopy, and Datacom. PCO engaged Unisys as its implementation partner to assist with the completion of the PAL project, in particular with the development of the technical solution and systems integration.
A legislation management system has some superficial similarity to any other office document management environment, but a number of factors distinguish legislation from other document environments, including the longevity of the documents, the importance of the documents, the regular structure of the documents, the temporal nature of the documents (the requirement to access current and old versions that reflect the effect of amendment documents as made into law), and the unique relationship between the drafters and the Parliament. In New Zealand, these are further complicated by the split functions of the PCO (as both drafter of legislation and publisher of legislation—a common development in many jurisdictions), and the time pressures placed on the drafters and publishers (because of the long sitting period and the extensive use of select committees). Producing a system that takes into account these factors is a complex task.
The PAL system affects a large number of key stakeholders and all of their interests must be balanced. The key driving force behind the project is to make legislation accessible to the New Zealand public. This can assist access to:
- the democratic process: by providing context to debate on new legislation and by assisting the Parliament and the Office of the Clerk to perform their functions,
- the judicial process: by reducing the cost of gaining access to the law both directly to the public and indirectly through legal publishers, and
- the executive process: by assisting the PCO and IRD to support the Government's legislative agenda, by reducing the cost to the public service of access to the law, and by improving the public's knowledge of the powers and obligations of the various officers of the Crown.
Access to the law means electronic access—to provide a fast and cost-effective delivery medium (hosted by Datacom and Parliamentary Service) and to assist the visually and physically impaired, and paper access—to fulfil the legislative requirements of the Parliament (typesetting provided by PCO and printing outsourced to SecuraCopy) and those who are unable to access electronic versions. It requires timely access to draft legislation (prepared by PCO, IRD, and OC), the output of select committees (prepared by OC and PCO), legislation as made (prepared by PCO), and consolidated legislation (prepared by PCO and Brookers).
Unisys, as implementation partner, has put its reputation on the line to deliver a system that supports the needs of the other stakeholders.
InQuirion has consulted widely amongst the stakeholders to ensure a complete and thorough coverage of the issues raised by this crucial project. These stakeholders cooperated willingly in providing documents, information, and time to facilitate the review. The process of the review included:
- an initial review of the system and project documentation,
- an initial on-site visit incorporating:
- focus groups and user interviews,
- demonstrations of the system—with and without direction from PCO and Unisys,
- a system architecture briefing, and
- formal and informal code reviews and walk-throughs,
- vendor interviews to ascertain future product directions,
- preparation and presentation of a draft report, and
- response to feedback to produce this final report.
1.5 Summary of recommendations
The system and the architecture behind it are generally sound. Although there are a number of issues identified in this document, InQuirion is confident that, if these issues are addressed together with those identified in the draft Stage 2 Phase 1 and Phase 2 Scoping document, New Zealand can be confident in deploying the PAL application. Below is a summary of the recommendations that come out of this review.
1.5.1 Reassess rendering engine choice
As described in section 6.2.2, InQuirion has some concerns about the current print rendering solution. These include both performance concerns—can it render large documents fast enough, and functionality concerns—can it render documents the way PCO and other stakeholders have specified. These concerns must be addressed before the system can be deployed.
InQuirion recommends that the User Acceptance Team prepare a set of documents in XML and PDF that can be provided to Unisys and ArborText for confirmation of their project plans and against which to validate candidate releases. This set of documents together with the layout specifications should be provided to selected vendors of alternative rendering engines to ensure that a fallback position is preserved and to explore the cost-effectiveness of alternative solutions.
Further details about the desired contents of that demonstration set appear in section 7.2.
1.5.2 Consider updating to Documentum 5
As described in section 6.3, there are potentially considerable advantages to upgrading from Documentum 4i to 5i for the Content Management System (CMS). These include considerable performance improvements with the throughput of XML documents that are chunked, or split into Parts and Subparts to allow multiple drafters to work on a single draft simultaneously, and better support for the rules that define when an element should be treated as a separate chunk (for instance, Parts inserted by amendment wording should probably not be in a separate chunk).
PCO should consider the feasibility of upgrading for Phase 1 release, and certainly for Phase 2 release. To test the feasibility of Phase 1 upgrade, PCO should install Documentum 5i on a separate machine and, either using their own IT staff or Unisys, with assistance from Documentum as required, attempt to install the PAL application to identify the issues and obstacles to migrating. If the install is successful, the User Acceptance Testing team (UAT) should be used to exercise the install to identify any components of the system that are not functioning as they should. This process should inform the decision on whether or not to upgrade before deployment.
As described in section 6.2.1, drafters and other users of the Authoring Tool are often editing documents with the tags showing. Because of the density of tags in the New Zealand PCO DTDs, the amount of text visible is quite small on smaller screens. Drafter and other user productivity is likely to drastically improve with no modifications to the software, if drafters and other users are provided with large, high-definition screens with which to prepare and edit drafts.
While these upgrades are not crucial to deployment and could take place at any time before or after deployment, obviously the productivity improvements will not be realized until the screens are installed on the users' desktops.
1.5.4 Use native Unicode for macrons throughout the system
As described in sections 6.5.1.2 and 6.5.4, Maori macrons are not being managed in the system in a way that supports data and application portability. InQuirion recommends these macrons and other characters beyond 127 in the Unicode code tables be managed as native Unicode rather than using entities. The Authoring Tool can be configured to import and export macrons using the native Unicode encoding. This will ensure that documents can be passed through any conformant XML tools and still be readily edited in Epic Editor. This change is best implemented before go live to avoid later clean-up tasks.
Should the New Zealand Government wish to deliver Maori macrons correctly on the website, it should be configured to send HTTP headers and metadata fields in the HTML identifying the encoding as UTF8 and native Unicode should be delivered in the web pages. A canonical decomposition of the Unicode data should be applied (to separate an "a" into "a" followed by the "-" combining diacritic) so that older web browsers incapable of displaying the macrons still display the vowels appropriately.
1.5.5 Consolidation of the Authoring Tool documentation
As described in section 6.10.1, the documentation of the Authoring Tool has been augmented by a number of separate documents for each new function added to the Authoring Tool with separate installation instructions for each new function. Ongoing maintenance of the Authoring Tool and the whole system would be aided by consolidating these documents into a single User Requirements Document—describing what the Authoring Tool and the various functions delivers to the user, Detailed Design Document—describing how the functionality was achieved in the coding environment, and Installation Manual—describing how to install the whole application as well as separately installing the additional functions in an existing install.
InQuirion believes that the system is not complete until it has been delivered in a maintainable form and that this consolidation of the documentation is necessary to deliver a maintainable application. If Unisys is responsible for operational maintenance between Phase 1 and Phase 2, delivery of this consolidated documentation could be deferred to Phase 2 deployment.
1.5.6 Make user acceptance testing team available throughout development
As described in sections 6.10.2 and 7.1, the User Acceptance Testing team (UAT) has developed considerable expertise in exercising candidate releases and identifying issues and reporting errors. InQuirion recommends that the UAT team continue to be used throughout the development for Phase 1 and Phase 2 to test candidate releases for Unisys and its subcontractors to reduce the risk that the final candidate release will be rejected by UAT.
For ongoing development, InQuirion recommends that the UAT team be regularly reconvened to test candidate releases of newly developed functionality as part of the ongoing enhancement of the PAL application.
1.5.7 Consider more redundancy for the systems critical for publication
As described in section 7.5, while there is considerable redundancy in the production environment within PCO, external access to the legislation is subject to a few key points of failure. Those accessing the internal repository through a Virtual Private Network (VPN) must access through a single piece of hardware to support that VPN. If that box were to fail, a replacement could take some time to procure. InQuirion recommends that arrangements be made (including possibly purchasing a spare) to ensure that such a hardware failure does not interrupt the ability of IRD, SecuraCopy, Brookers, or Datacom to interact with the PCO repositories in the appropriate ways.
The public web site as hosted by Datacom sits on a single machine. While the original project envisaged relocating the testing machine from PCO to Datacom as the back-up machine, this machine will continue to be needed for ongoing testing. The PCO should either contract with Datacom to arrange a short-term replacement machine, or provide a warm backup machine to ensure that any hardware failure in the web server machine does not interrupt public access to the web site for more than an acceptable period of time. Any purchase of a new server should be deferred as late as possible to maximize value for money. This purchase should initiate a reshuffle of the hardware to ensure that the most powerful servers are deployed where performance is most critical to meet the objectives of the PAL system. The most likely candidates are to support print rendering or to support the public web site.
1.5.8 Consider upgrading the web engine to include an XML repository capability
As described in section 6.4.4, the web application that forms part of the PAL system is a simple, robust solution. But search is limited to full text and metadata fields extracted into the HTML data and fails to take full advantage of the additional value contained in, and potential of, the XML markup. For ongoing development, InQuirion recommends that the PCO explores using an XML repository linked to a web server or application server to deliver more extensive XML-based search capabilities, the ability to dynamically generate HTML appropriate for different browser versions from the XML, and support for point-in-time access to the consolidated legislation.
The current website architecture is adequate to meet the current requirements (although some outstanding issues need to be addressed before deployment). Because the web site is an output or delivery mechanism only, the impact of changes in the web site to the production environment will be minimal. Upgrades could occur before, during, or after deployment of the production environment.
The PAL system as implemented contains a number of strengths outlined in this report. The weaknesses described in this report are not insurmountable and should be addressed by implementing these recommendations and the draft Stage 2 Phase 1 and Phase 2 Scoping document.
Generally, the integrity of the modules has been preserved with customizations appropriate to the applications on which they have been built.
The modularity of the system follows similar systems deployed or in development around the world. While there are still some outstanding issues to do with integration of the system components, the system components seem to be interacting well and reliably.
Scope remains within the applications for further customization and upgrade of system components. With such a large and complex software system, the impact of any upgrades or enhancements will always need to be thoroughly tested before deployment but this is not unusual for developments of this type.
The use of industry standards and coding practices is evident within the development. The variety of subcontractors and development environments deployed on the project has lead to inconsistency between coding practices in different modules. Again, this is not unusual for projects of this size and complexity. Some minor recommendations have been made for improving the application of standards and coding practices throughout this report.
The system is generally robust. At no time during demonstrations of the system to InQuirion or InQuirion interacting with the system did any of the components crash although there were problems with some of the outputs of the modules.
Before deployment, a number of issues relating to ongoing maintenance and support need to be addressed. These have been outlined in this report.
Providing these issues and other issues identified within this report are addressed to the satisfaction of the stakeholders, the New Zealand Government can be assured that the PAL system, when implemented, will be operationally stable, maintainable, and capable of supporting future enhancement and development.
2. Introduction
From the Terms of Reference for this Technical Review:
The Parliamentary Counsel Office (PCO), in conjunction with the Office of the Clerk (OC) and the Inland Revenue Department (IRD), is undertaking the Public Access to Legislation project (the PAL Project). The project is designed to improve the way in which all New Zealand legislation, drafted by PCO, OC, and IRD, is made available, and to provide the basis for public access to up- to-date legislation in both printed and electronic form. The project scope covers authoring and pre-publication tools; content management system; Document Type Definitions (DTDs); database of New Zealand legislation; website and other electronic access solution; change management and communication; systems integration; secure connectivity between the PCO and other agencies (OC, IRD, Brookers Ltd, and Securacopy); establishment of capability for pre-publication, compilation, and database officialisation; and arrangements for printing of hard copy output. (See: www.pco.parliament.govt.nz).
PCO engaged Unisys New Zealand (Unisys) in 2001 as its implementation partner to assist with the completion of the PAL Project, in particular development of the technical solution and systems integration.
The PAL system will operate over a virtual private network (VPN) and, on implementation, will support around 80 users at PCO, OC, IRD and Brookers Ltd.
A web site will support public and subscriber access to legislation in XML, HTML, and PDF format. SecuraCopy, a contract printer, will perform the commercial printing of legislation.
The project was commissioned over two stages:
- Stage 1, covering Planning, Scoping, and Analysis, and the evaluation and selection of system components, was completed at the end of 2001, and
- Stage 2, Implementation, began in March 2002.
Following several plan revisions during 2002, the parties agreed a go-live date of 17 February 2003. The planned Go-live date has been deferred, for a number of technical and commercial reasons. The PCO and Unisys have subsequently produced a revised statement of requirements for the completion of Stage 2. The PCO now wishes to commission a technical review of the PAL solution before the New Zealand Government makes any decisions about the future of the project.
InQuirion Pty Ltd was engaged by the New Zealand PCO to undertake this technical review in order to inform decisions about the future of the project. The primary role of the review is to enable the PCO to advise the New Zealand Government, as sponsors of the PAL project, whether the PAL system, if implemented, would be operationally stable, maintainable, and capable of supporting future enhancement and development.
The lead consultant on this project, Dr Timothy Arnold-Moore, has extensive experience in developing complex SGML and XML-based solutions for legislative drafting and has consulted to the governments of Tasmania, Canada (federal), New South Wales, Ontario, and Papua New Guinea on legislative drafting solutions. Neal Landry, Neda Miskov, and Michael Fuller assisted him.
2.2 A construction project analogy
Producing an information technology (IT) system has many similarities to constructing a building. A small renovation might require only limited planning and one or two skilled technicians a week as with a small change to an existing IT system. Larger changes like adding a second storey might involve more specialized skills, more detailed planning and coordination, and a longer time frame.
Building a house is a well-understood problem and many firms, large and small, can provide a service with varying degrees of customization and cost as with well-understood IT functionality like pay-roll systems. In both environments there are a wide variety of "off- the-shelf" solutions available. Predicting the cost and time necessary for such a project should be and usually is relatively simple.
On the other hand, large public building projects usually require considerable custom design and construction. Rather than a small construction firm, such projects usually involve a reputable architectural firm and are typically built by one of a few companies specializing in large-scale projects. Similarly, there are a small number of organizations that specialize in large-scale public IT projects. Unisys is one such company that has developed a well-deserved reputation for successfully completing large-scale public IT systems. Like successful large-scale construction companies, Unisys has built many large IT systems for government and large corporate clients and has developed methodologies based on considerable experience in large-scale IT project development.
In construction projects, the architect is employed to represent the client's interests with the construction firm, to ensure that the client's requirements are fully understood, to produce designs that capture all of the client's requirements, and to ensure that building commences and continues on a design that meets the client's needs and is signed- off by the client. Architecture is a well-understood discipline that encapsulates centuries of experience in construction. In the medieval period, an architect or construction firm was employed to build a cathedral if the last one built didn't fall down. Now architecture courses ensure that qualified architects combine technical knowledge (about how to build buildings that stay up) with the ability to elicit and understand the clients' needs, the experience to design constructions that satisfy those needs, and the management skills to plan and coordinate their construction.
Designing and building software systems is a much newer discipline. It is rare to find a single person with the technical skills, the business analysis skills (both interacting with potential users and output sources), and the project management skills to perform the equivalent task as an architect for a software project. These tasks are often divided between a project manager, a technical architect, and one or more business or document analysts and this has been the case with the Unisys team. Even though courses in software engineering exist, the reasons for project success or failure are still not well understood. While the software industry is not medieval, often the safest way to ensure a successful IT project is to employ a firm that has built a system like the proposed system before, or for the firm to involve in the role of architect someone with experience in building systems of that kind. Based on InQuirion's experience in producing end-to-end legislation systems, InQuirion considers a critical success factor in such projects is the involvement of someone with that kind of experience as architect.
2.3 Why legislation is different
A legislation drafting and management system on the face of it looks like a typical office document authoring, management, and delivery application. But there are a number of factors related to legislation that distinguish it from typical office environments. These include:
- The longevity of the documents: Some legislation being managed today was drafted many hundreds of years ago—portions of the Magna Carta are still part of the law in New Zealand. By contrast, most office documents are created for immediate use. Few are retained for more than a year or two.
- The importance of the documents: After the Treaty of Waitangi, the documents recording the legislation of New Zealand are arguably the most important documents in the land. 1 This importance justifies particular care in ensuring the highest quality of print production when rendering the documents. The main driver for office documents is efficiency. Providing that the documents look sufficiently professional, minor variations in typography can be overlooked. While efficiency and timeliness are important for legislation, presentation is particularly important and variations in typography, however small may be noticed and trigger unnecessary delays in the Parliamentary process.
- The regular structure of the documents: Legislation in New Zealand, as in most English-speaking jurisdictions, follows largely the English tradition breaking Acts into numbered sections, subsections, paragraphs, and subparagraphs each with distinctive numbering conventions and typographic markers and consistent nomenclature, and grouping the sections into Parts and Subparts and under various unnumbered headings. While some of the typographic markers have changed subtly over the years and vary from jurisdiction to jurisdiction, the structure is largely unchanged and uniform across most English- speaking jurisdictions. Office documents vary much more widely in structure.
- Temporal nature of the documents: The vast majority of legislation made in New Zealand, as in most English-speaking jurisdictions, is amending legislation—that is the wording of the legislation describes textual changes to existing legislation. While these amendments have the force of law, users of legislation care more about the wording that results from applying the amendments to the existing substantive provisions (referred to as "consolidating" the amendments hence the term "consolidation" to describe these documents) than the actual text describing the amendment. This results in multiple versions of the substantive legislation and different versions will be valid at different times. Users of legislation require immediate access to the most current consolidation of a piece of legislation but they also need access to past versions—when preparing legal advice or hearing a case about a past incident—and, to the extent possible, future versions—when preparing advice about future activity. Office documents rarely exist long enough for multiple versions to be valid and, where multiple versions exist, usually only the latest version is of interest. The time period of validity of most office documents is rarely as clearly defined as for legislation.
- The unique relationship between the drafters and the Parliament: The drafters provide legal advice to the Parliament and owe fiduciary duties, including a duty of confidence, to their clients—the Members of Parliament and the officers of the Crown. The documents, while being drafted, are highly confidential and may contain politically sensitive (and occasionally militarily sensitive) information. Once the drafts are tabled, they are public documents. In a normal office environment, most documents either stay sensitive throughout their life cycle or are designed from the beginning for public distribution.
- Split functions of the PCO: Drafting offices such as the PCO have existed for many years solely to draft bills and associated material and regulations and other subordinate legislation for the government. As in many other jurisdictions, the NZ PCO is also now responsible for publishing the legislation and related materials. This is sensible because the PCO is uniquely positioned to maintain many of the products useful to the broader public for its own internal use (including up-to-date consolidations on which to base amendment Bills or draft Regulations). A culture of providing a discrete and largely confidential service to the Government does not always sit well with the more public service of publishing.
- Time pressures placed on the PCO, OC, and IRD: New Zealand Parliament has an extremely long sitting season. Parliament rises a little before Christmas and resumes in early February. With the exception of this 5-6 week break, there are only 3 or 4 breaks of no more than 3 weeks scattered throughout the year. There is little opportunity for respite for the services that support the operation of Parliament including the PCO, OC, and IRD drafting. Unlike many Westminster Parliaments, where a government can often predict with considerable certainty what the final form of an Act will be well advance of its third reading or Assent, the extensive use of committees within the New Zealand parliamentary process means that changes in the text of a Bill are common and less predictable. Committees may work on reports to the Parliament up until a day or two before they are due to be presented, and Assent to a Bill follows on fairly quickly after third reading. Therefore, the OC and the PCO are under considerable pressure to prepare paper versions of a Bill or Act very quickly. Other jurisdictions may be able to pre-prepare Bills or Acts with some certainty many days or even weeks before they are required but this is not often the case in New Zealand.
These features add to the complexity of constructing a legislation drafting and management environment. They require both unique features and novel development techniques to ensure a lasting and usable system.
3. Stakeholders
This section describes the various stakeholders in the outcome of the PAL project. It does not attempt to comment on the extent to which the existing PAL implementation satisfies the requirements of those stakeholders. This section aims to provide a succinct statement of the dominant interests of those parties with respect to this project.
The PAL project is driven by one major overriding force—that is to make New Zealand legislation accessible. Since 1989, the New Zealand Statute Book has been owned or controlled by private corporations and complete sets of consolidations were only available from commercial legal publishers for a fee.
In a democratic government—where the citizens have a voice in the creation and modification of the laws that govern them—it is fundamental for the correct operation of the legislature that all citizens have access to the text of the law so that their influence is informed.
In a common law jurisdiction—where ignorance of the law is no excuse—it is fundamental to correct operation of the judiciary that all citizens have equal access to the text of the law so that all citizens, criminal or law-abiding can readily discover their obligations and rights under the law.
In a welfare state—where taxation funds a wide variety of services and wealth redistribution—it is fundamental to the correct operation of the executive that all citizens have ready access to the legislation that creates and guides the exercise of the powers and obligations of the public service to ensure that citizens are receiving the intended benefit of the services provided.
Effective access to the law by the citizens of New Zealand must be:
- complete—it must include the entire statute book, including all Acts and Regulations as made, and to support the legislative process, Bills and proposed amendments to Bills (including SOPs), and consolidations, to ensure an authoritative source of the text of the law as in force at a particular time,
- timely—it must be available as soon as is practicable after it becomes law, and in the case of Bills, SOPs and slip amendments, for a reasonable time during the parliamentary process to ensure adequate opportunity to lobby elected representatives, and
- accessible—it must be available to citizens with disabilities, whether visual impairments (rendering paper publications inaccessible), physical impairments (rendering centralized distribution facilities such as government information kiosks or government bookshops inaccessible), or technological impairments (rendering web-only delivery inaccessible to those who require traditional paper publications).
3.2 The Government of New Zealand
The primary historical role of the PCO is to support the government's legislative agenda by providing a timely, high-quality drafting service for primary (Bills, slips and SOPs) and subordinate legislation. The government want to ensure that the new system is introduced without adversely affecting the legislative timetable, either by slowing down the current drafting service or turn-around time, or by introducing minor formatting errors that distract the Members from the task of debating the content of the legislation. Despite acquiring new responsibilities as custodians and publishers of the Statute Book, the PCO must continue to provide a high level of service to the government.
The PCO, the OC, and Legislation Direct between them have in the past provided a service to mark up committee amendments, split Bills, and undertake other potentially time consuming tasks in very quick turn around times. As PCO assumes more of the responsibilities formerly taken by Legislation Direct, it is imperative that they provide as good or better service as provided in the past so that the legislative process goes smoothly and Members have available to them timely information for consideration before debating or voting.
3.4 Parliamentary Counsel Office (PCO)
The primary sponsor of the PAL project is the Parliamentary Counsel Office. The PAL system needs to support the historical role of the PCO in delivering services to the government and Parliament, and the new role of the PCO as custodian of the Statute Book and publisher of legislation.
PCO drafters will spend most of their time working in the Authoring Tool fulfilling their primary function of drafting proposed legislation or amendments to legislation or drafts for the government. While the primary focus of the PAL system has been to provide public access to the outputs of the drafters, for the PAL system to be truly effective, it must deliver this public access without interfering with or adversely affecting the task of drafting the legislation.
3.4.2 Secretaries and support staff
The secretaries and support staff work with drafters to support the production of draft legislation and amendments. The PAL system will be most effective if it avoids imposing additional burdens on the secretaries and support staff but instead provides useful tools to improve the productivity or quality of their output
Because of the high importance of the legislative documents and the potential costs, political and financial, of mistakes in the legislative documents, like most jurisdictions, New Zealand engages proofreaders to edit and correct draft legislation and other documents to ensure that the documents that leave the office are of the highest possible quality. The PAL system must continue to support the role of the proofreaders, making the drafts available to the proofreaders immediately that the drafter requests a proofread, and making their comments and suggestions available to the drafters immediately on the proofreaders completing their task.
The Pre-Publication Unit will be taking on the publication role formerly undertaken by Legislation Direct. They will bear the primary responsibility for ensuring that the correct stylesheets are applied to the correct documents, that the output of the print process produces high-quality page presentation, and that the documents are made available in a timely fashion, both in print and electronic form, once the content has been created. They will also be responsible for managing the more unusual inclusions in legislation such as graphics, treaties and deeds, and complex tables. A well-crafted print and electronic rendering solution as part of the PAL system should eliminate much of the stress of this typesetting role and allow the PPU to concentrate on the presentation of the more unusual elements.
One of the objectives of the PAL project is to make not only sessional or as made legislation available to the public, but also timely consolidations of amendments to supplement the official reprints. The PCO are engaging Brookers, the provider of the consolidated statute book, to maintain the consolidations while PCO staff are validating the consolidations to facilitate the officialization of the consolidated statute book. Brookers staff will be housed both within the PCO and on Brookers premises connected by a Virtual Private Network (VPN). When all of the consolidations are officialized, the Reprints Unit (RU) within PCO will assume responsibility for maintaining the consolidations.
The PAL project needs to provide tools to support the compilers, whether working on official or unofficial consolidations, to allow them to track all amendments and make sure that they are applied at the right time in the right way, to ensure that appropriate history notes are generated and to assist with the generation of the various publications that report on the commencements and amendments in particular time periods or applied to particular legislation.
The IT Team's role is to contribute to the effectiveness and efficiency of the PCO by assisting with the reliable operation, effective use, and development of the PCO's computer systems.
The PAL system will be most effective if it integrates seamlessly into the current IT infrastructure, and if the current high level of user satisfaction continues.
3.5 Inland Revenue Department (IRD) Drafters
In addition to the legislative drafters in PCO, Inland Revenue Department (IRD) has a small team of drafters dedicated to drafting tax and associated legislation. Their requirements are similar to those of the PCO drafters, except they will be accessing the repository of legislation and work in progress from IRD's offices across a virtual private network (VPN). It is important for their productivity that the network provides a reliable connection and sufficient bandwidth so that they do not have to wait very much longer to check-in documents or request and receive print previews or other renditions than users physically located in the PCO.
The Office of the Clerk (OC) is responsible for supporting the business of the House. They manage the progress of Bills through the Parliament and manage and record the decisions of the Parliament and reports from Select Committees to the Parliament. They need to be able to access the print proofs of documents to ensure that they match the decisions made by the House (and electronic versions to ensure no changes have been introduced in unamended text) and need a responsive print publication service from the PPU for the preparation of the various versions of the Bills for each stage of the legislative process, and assent copies to bring the Bills into law. Since PCO drafters are only available to the Government, private members Bills are drafted by OC.
SecuraCopy is a private firm that is effectively taking the role of an outsourced government printer. In the new PAL system, the typographic page setting work will be done primarily by the system and managed by the PCO's PPU. SecuraCopy will be responsible for preparing the print versions from PDF documents and an XML electronic "envelope" that gives instructions about what should be done with each PDF—how many copies and to whom to send them. SecuraCopy will also be responsible for distributing the print copies through normal commercial channels. It is most important to SecuraCopy that the PAL system provide a reliable connection for the communication of the proofs and that the instructions about volume and distribution are correct so that the right documents get sent to the right people at the right time.
SecuraCopy is connected to the PCO's network via dark fibre to ensure that communications between SecuraCopy and PCO are secure (that when a request is received by SecuraCopy that it is actually sent by PCO) and the confidentialities encapsulated in the more sensitive documents are not breached. Replistor technology is used to transfer documents.
Just as SecuraCopy provide an outsourced distribution channel for paper delivery, Datacom, as the Internet Service Provider (ISP), provide the outsourced distribution channel for electronic delivery of legislation. Datacom host the machines responsible for the public website as well as the service for downloading XML and PDF updates, see section 3.10 below. Updates to these services are uploaded on a daily basis (or by request) through a VPN using the Replistor technology to communicate only those files that have changed since the last replication.
The Parliamentary Service, amongst other things, is responsible for providing the network infrastructure to both PCO and OC. It provides firewall services to prevent unwanted intruders from breaching these sensitive networks, and hosts the PCO end of the VPN services to limit outside access. Its main interest in PAL is to protect the integrity of the network service to PCO, OC, and the other agencies that share their network infrastructure.
In addition to the public website that will provide HTML-based search and browse capability of as made and consolidated legislation, the PCO is also making available a free subscriber service to sessional and consolidated legislation in XML and PDF form, primarily for legal publishers. This service will maintain the last month's as made and consolidated documents so that third party publishers can access authoritative XML and PDF versions of New Zealand legislation for repurposing and republishing.
Unisys, as implementation partner on the PAL project, has a significant financial and reputational investment in the PAL project. While this interest is likely to diminish over time, they have a current interest in ensuring the successful deployment of the PAL system, and a potential continuing interest in securing an ongoing operational and developmental maintenance contract for the PAL system.
The legislative process is complex and adopting a centralized system for managing the total document lifecycle of legislative documents requires co-ordination of the needs of a large number of different interest groups and stakeholders to ensure that a complete system is successfully deployed.
4. Process
Given the large number of organizations and requirements for the PAL system, it was necessary for this review to consult widely to ensure a complete and thorough coverage of all the issues raised by this crucial project. The efficiency of this process largely relied on the cooperation of the various stakeholders in providing information to InQuirion in the form of documents, code, and interviews during a short but intensive on-site period and with subsequent phone and email clarifications and confirmations. InQuirion would like to state at the outset that all of the stakeholders, including Unisys, the vendors, and subcontractors, were extremely cooperative and helpful with InQuirion personnel during the course of the technical review. The following summarizes the processes followed to identify the issues of relevance to the PCO and other stakeholders.
4.1 Initial review of system documentation
Prior to visiting the PCO offices in Wellington, InQuirion was provided with extensive system and project documentation including the following documents:
| Parliamentary Counsel Office |
|
| PCO Test Scripts Website Test Scripts UAT Plan UAT Process UAT Requirements Elements of Legislation Specification DTD QA Report by Complete Data Solutions Ltd |
|
| Unisys
|
|
| Content
Management System |
CMS Design Document v1.1 CMS Configuration and Customisation Guide v1.1 Transform Process Technical Documentation v1.1 XSL System Documentation v1.0 Metadata Guidelines v0.2 Metadata Specification v1.1 |
| Database |
Legislative Database—Markup & Re-purposing Specification v1.4 |
| Document
Type Definitions |
DTD History v1.2 DTD Design Specification v2.1 DTD Guide v2.9 ID Schema v1.41 |
| Website |
PCO Website Detailed Design v3.0 PCO Website Style Guide v3.0 Website UI Design and Supplementary Specification Website System Documentation Subscribers Website Configuration Indexing Options |
| Systems
Integration |
Book of Requirements & Functional Specifications Technical Architecture Plan v1.1 Remote Access Architecture v0.6 Remote Access Detailed Design Document v1.2 PAL VPN Client Software Configuration v1.2 Stage 2 Phase 1 and Phase 2 Scoping Statement 1.0 Server Documentation Callisto v1.1 Server Documentation Europa v1.1 Server Documentation Jupiter v1.1 Server Documentation Leda v1.1 Server Documentation Leo v1.1 CMS Standby Server Configuration Guide v.1.0 Legato Replistor System Documentation v1.0 Interface between PCO and SecuraCopy Configuration v1.1 Graphics Features v1.1 Training Materials PCO PPU |
| Authoring
Tool |
ADG Autotext v1.0.doc Authoring Tool—Custom Fonts.doc Authoring Tool End User Specification v1.0 Final.doc Authoring Tool Matrix 7 August 2002 v1.0 Final.xls Authoring Tool Workstation Installation Notes 1.1 with track.doc Epic Authoring Application v1.0.doc Epic Editor 4.4 release notes (4.3.1 start at chapter 4).pdf NZ PCO Autonumbering Utility Installation and User Guide v1.doc NZ PCO ID Schema Utility Installation Guide v1.2.doc NZ PCO Initial Startup Dialog v1.4.doc NZ PCO Licence Release Utility Installation and User Guide v.doc NZ PCO Numbering Restart Utility Installation and User Guide.doc NZ PCO Publish Utility for Drafters Installation and User Gu.doc NZ PCO Publish Utility for PPU Installation and User Guide v.doc NZ PCO Return Quick Key Installation and User Guide v1.2.doc NZ PCO Stylesheet Movement Utility Installation and User Guide.doc NZ PCO_Startup DialogV1.4.doc Revision Tracking Tool Installation and User Guide v1.4.doc |
After an initial familiarization with the system via the documentation, InQuirion then commenced an on-site visit in Wellington. InQuirion was keen to elicit information from as many stakeholders and participants in the PAL project as possible without jeopardizing the timeliness of the review.
4.2.1 Focus groups and user interviews
A number of different focus groups and interviews were convened in order to explore and understand any reservations different categories of user have about the PAL system and to elicit details about any perceived or actual limitations of the system. The review was commenced by a focus group representing PCO (including drafters, secretaries, IT, and PPU), OC, IRD, and Unisys. A summary of the other interviews and focus groups appears in the table below in section 4.2.5.
The bulk of the first day on site was taken up by an initial system demonstration. The User Acceptance Testing Team (primarily Melanie Bromley—a drafter, and Michelle Antoine—the head of PPU) led the demonstration. This included exploring the full document life cycle of a Bill from initial draft through to publication as an Act. While there were a number of formatting issues visible throughout the demonstration (including not being able to generate Bills with line numbers), the demonstration showed support for the bulk of the lifecycle—however the demonstration was designed to avoid areas of the system where known defects prevented completion of a document lifecycle. Throughout the onsite visit, other parts of the PAL system and the existing systems used by PCO to manage the drafting process were also demonstrated to InQuirion.
4.2.3 System architecture briefing
In order to augment and explain the system documentation, InQuirion required a system architecture briefing from the PCO technical staff and then from Unisys. The purpose of this briefing was both to explore the extent to which the developed system matched the architecture documentation and to ascertain the depth of knowledge the various personnel had of the overall system architecture. InQuirion is satisfied that the developed system substantially corresponds with the architecture documentation. While PCO technical staff demonstrated a good overall understanding of the system architecture and how the pieces fitted together, Unisys was not able to present one person who both understood the architecture and the individual technologies, and understood the business requirements of a legislative drafting office. The knowledge seemed to be split between numerous Unisys staff and subcontractors with no single system architect uniting the vision.
4.2.4 Code reviews and walk-throughs
A thorough technical review of a project of this size and complexity requires an examination of the code to implement it to ensure consistency and quality. In addition to simply looking at the code, InQuirion had members of the development team—Unisys employees and subcontractors—walk them through a number of areas of the code, explaining the design decisions and layout of the code. While time prevented InQuirion from reviewing every single line of code, a combination of formal and informal reviews with the aid of developers and InQuirion staff examining the code directly without assistance ensured thorough coverage. There was a focus on the areas of code of most risk, and immediate response by the developers on any code or practices perceived by InQuirion to be unusual.
4.2.5 Summary of the on-site schedule
Date |
Purpose |
Participants |
| WEEK 1 |
||
| Monday 18th | Initial meeting | Geoff Lawn |
| Team meeting | PCO Team Alan Grainer OC IRD |
|
| System demonstration | PCO Team (primarily Melanie Bromley and Michelle Antoine) | |
| Tuesday 19th | Review user issues |
PCO Team |
| Systems architecture review | PCO Team, Judy Heaphy, Devon Heaphy, Tim Woodill | |
| Unisys architecture review | Unisys (Alan Grainer, Dermot O'Brien, Lesley Jones, Lee Shelton) |
|
| Wednesday 20th | Website review (including code review) |
Unisys (Helen Hunt, Todd Mansill) |
| Transforms review | Unisys (Ben Horan, Lesley Jones) | |
| CMS review | Unisys (Lesley Jones) | |
| Thursday 21st | Publishing issues |
Brookers (Mark Bacon, Jenny Sinclair, Jane Pilkington) |
| DTD review | e-Gloo (Alan Burton) Brookers (Mark Bacon) | |
| Stakeholder interviews | Office of the Clerk (Fay Paterson, Donna Tunnicliffe) Securacopy (Chris Eales) IRD (Warren Cole, Margaret Nixon) Datacom (Mark Muru, Sarah Weavers) |
|
| Website code review | Unisys (Helen Hunt, Todd Mansill) | |
| Transforms code review | Subcontractors (Bevan Souster, Mike Player) | |
| Friday 22nd | Authoring tool users focus group |
PCO, OC and IRD |
| Stakeholder interview | Parliamentary Service (John Preval) | |
| Following week planning | Geoff Lawn, Anthony Baker, Alan Grainer | |
| Date |
Purpose |
Participants |
| WEEK 2 |
||
| Monday 25th | Progress meeting |
Geoff Lawn, |
| Authoring Tool and rendering engine issues | Alan Grainer ADG (Tammy Halter) | |
| Office of the Clerk issues and background | Donna Tunnicliffe | |
| Print production unit | PPU Team | |
| Tuesday 26th | Code reviews |
- |
| System familiarization | - | |
| Transform issues | Unisys (Lesley Jones Mike Player) | |
| Wednesday 27th | Stage 2 Scoping Document review |
PCO, |
| Systems Testing review | Unisys (Lee Shelton) | |
| Thursday 28th | Stage 2 Scoping Document review ct'd |
PCO, |
| Future enhancements | PCO Team | |
| CMS Technical review | Unisys (Alan Grainer, Tao Zhang) Documentum (Mike Pomponio, Debra Bordignon) | |
| Friday 29th | Legislation tracking |
PPU |
| Support issues | PCO, OC and IRD | |
| Documentum future directions | Unisys (Alan Grainer) Documentum | |
| Review meeting | Geoff Lawn, Anthony Baker | |
In order to assess the ongoing viability of the PAL project and ensure that it is maintainable into the future, it was necessary to interview representatives of the vendors of the main components of the PAL system, ArborText—vendors of Epic Editor and the Epic Print Composer and Epic E3 rendering engines—and Documentum—vendors of the content management system. The vendors were very cooperative in providing access to local representatives as well as senior executive and technical staff in order to provide all the relevant information about future product directions.
The findings that InQuirion makes in this area rely on the information provided to it by the vendors. PCO should be aware that the IT market can change very quickly and even plans outlined by the vendors in good faith to InQuirion may change drastically to accommodate unforseen market developments.
4.4 Preparation of draft report
Subsequent to the on-site visit, InQuirion has spent a number of weeks collating the information gathered from the documentation and the various interviews and focus groups and code examinations to form an opinion on the strengths and weaknesses of the developed system, the integrity of the customizations and the core products, the integration and modularity of the solution components, the scope for future customization and upgrade, the use of industry standards and coding practices, the overall robustness of the solution, and any implications for ongoing operational management and support. These findings and recommendations have been collected together to form this report.
4.5 Presentation of draft report
After delivery of the draft report to the PCO, InQuirion presented the report to the various stakeholders in Wellington on October 1st. InQuirion's lead consultant remained in Wellington for the rest of that week in order to elicit feedback on the draft report and include any changes.
A second draft was delivered to PCO and additional feedback from the stakeholders was provided to InQuirion.
5. Architecture
5.1 Typical architecture for legislative drafting
There have been a number of efforts throughout the English- speaking world to apply SGML and XML technology to the task of legislative drafting, management, and publication. During the information-gathering phase, New Zealand PCO consulted a number of drafting offices in jurisdictions that had either commenced or completed projects to implement these technologies to learn from their experience and avoid their most obvious mistakes. There has been a considerable push amongst Anglo-American jurisdictions for higher quality electronic access to the law and more timely availability of the legislative artefacts. The catch-cry of e-government is not just a political tool—it has the potential to deliver real benefits to the community and enhance the democratic process valued throughout the common law world.
Common to all of these projects are a number of fundamental components. Not all jurisdictions have used technological solutions for each of the components, but each has developed a solution that fits their requirements and budgets.
The intent of a drafting system is to provide a set of tools to support the authoring of legislative documents. That is the primary task of the drafters and the tool in which they author drafts is the most fundamental piece of the solution. A number of different approaches exist. Some jurisdictions have adopted a structured authoring environment that validates SGML or XML drafts as they are created such as ArborText Epic Editor (Canada Department of Justice, New Zealand PCO) or Corel XMetaL (US House of Congress, California, Canada Parliamentary Counsel). Some have adopted a word processor environment for the drafters with down stream processing by additional staff (Singapore using WordPerfect and then FrameMaker+SGML, Western Australia using Word and a combination of SGML tools, Ontario using Word and in-house conversion, Quebec with Word and IroSGML) or interactive conversion on demand (Tasmania and PNG with Word and TeraText for Legislation, NSW PCO with FrameMaker native and FrameMaker+SGML).
The next most important piece of the architecture is the repository for managing multiple versions of the drafts and the statute book. Many jurisdictions have opted for the simpler solution at least in the short term of using the file system as the repository (Singapore, WA, US House, NSW although moving towards a repository technology). Others have used native XML repositories (Tasmania, PNG, Canada Department of Justice, and NSW using TeraText DBS) or traditional relational database systems (California, Quebec using Oracle, Canada House using MS SQL) or document management systems (New Zealand with Documentum, Tasmania and PNG with TeraText DMS).
5.1.3 Workflow
Legislative processes are complex with a large number of steps and the drafting processes that support them involve numerous additional quality assurance steps before entering the public legislative process. Jurisdictions need to track the progress of a file through these processes. Some do it with paper folders (Canada Department of Justice) or with simple database applications (NSW, Quebec). Others have adopted lifecycle support (New Zealand with Documentum) or full Workflow Management Coalition (WfMC) model workflow systems that are part of the repository technology (Tasmania and PNG with TeraText for Legislation, Hong Kong with Lotus Notes).
For most jurisdictions, the consolidation process is manual and typically uses either the same tool as for authoring (New Zealand) or a slightly more sophisticated authoring tool (Singapore, NSW PCO use XMetal for consolidation). Other jurisdictions have adopted a semi- automated (Canada Department of Justice and Quebec with IroSGML) or fully automated (Tasmania and PNG with TeraText for Legislation) consolidation system.
Many users of legislation are fairly conservative and print publications are still an important part of the output process. In most jurisdictions, the legislative process still relies on paper artefacts. Numerous tools are being used by Westminster governments to render the legislation into print format including low-end solutions (Tasmania, PNG mapping XML to Word), medium-cost solutions linked to the editing environment (Singapore, NSW using FrameMaker, New Zealand using Print Composer) and high-end solutions (Canada using 3B2).
One of the major motivating forces for adopting an electronic legislative drafting environment is to provide for more complete electronic delivery solutions, particularly web-based searching and browsing. The level of sophistication of the web delivery solutions ranges from generating static HTML for delivery out of standard web servers (Ontario and New Zealand using XSLT to generate HTML for IIS), through dynamic generation of HTML content direct from the repository (Hong Kong using Lotus Domino) including delivery of the XML to the drafters (NSW and Canada using TeraText DBS) to full point-in-time with all of these services (Tasmania using TeraText for Legislation).
5.2.1 Authoring/Consolidation tool
New Zealand has selected ArborText Epic Editor as the Authoring Tool for PAL. This tool is also the basis of the consolidation tool, with a manual consolidation approach being adopted in the short to medium term. A template for Microsoft Word 2000 has also been developed to support authoring of commentaries by OC, and the Word interchange capability within Epic Editor is used to translate this into appropriate XML.
Because New Zealand PCO has been using document management systems for some time and therefore understands the benefits associated with formal document management, access control, version management, and workflow/lifecycle control, it has selected a major DMS solution, Documentum 4i, as the repository for managing the multiple versions of legislation, draft and published, and for controlling access to those versions.
The advantage of choosing a major DMS such as Documentum for the repository is that workflow comes with it as a bonus. New Zealand has chosen to utilize the more flexible document lifecycle model supported by Documentum rather than a full Workflow Management Coalition workflow enactment service. The more complete workflow capability may be used later to support the functions currently managed by the Legislation Tracking System, a Microsoft Access application that will continue in use after PAL goes live.
Leveraging the stylesheet work needed for configuring the Authoring Tool, New Zealand has chosen a print-rendering tool from the same vendor as Epic Editor, ArborText's Epic Print Composer. This is a medium-level rendering tool that provides support for a number of industry standards including FOSI and XSL-FO.
New Zealand has opted for a fairly low-tech, low-risk web delivery solution using static HTML generated by exporting the XML out of Documentum through XSLT stylesheets. A relatively simple website implemented on MS IIS with ASP.NET infrastructure provides the web logic and dtSearch supports a simple set of searches over the text of the HTML and the metadata stored in HTML-specfic metadata tags.
5.3 PAL development environments
A number of different development environments and languages have been used to develop the PAL system.
Formatting Output Specification Instances (FOSIs) are used in Print Composer and Epic Editor to describe how to render the XML for display and print. This standard is an old Defence standard,2 designed for SGML that allows a developer to describe the presentation of each element and attribute. Like the W3C's CSS standard, FOSIs have limited capability to present content occurring once in the document in multiple places in the document (e.g. the title on the title page and in the running header) or to reorder the content for presentation. Although this is an open standard, there are few vendors still supporting it (primarily ArborText and Datalogics). Most have migrated to the W3C standards CSS3 or XSL-FO.4
ACL is used in Epic Editor to customize the operations of the editor and to provide legislation- and DTD-specific capabilities to the users of the Authoring Tool. ACL is a proprietary scripting language supported only by ArborText products. ACL customizations are collected in one or more packages. Epic Editor also supports scripting in VBScript, Java, and JavaScript (although none of them were used in customizing the Authoring Tool, despite the use of similar languages elsewhere in the project).
DocBasic has been used for customizing the actions when promoting or demoting a document in a Documentum lifecycle. DocBasic is a proprietary language supported only by Documentum, but it is based on Microsoft's VisualBasic, which, while also proprietary, is very widely used. Documentum does not embed VisualBasic into the Documentum product but an experienced VisualBasic programmer would find the code familiar and easy to modify or extend, although without the richness of the Visual Studio development environment.
DocBasic provides a number of APIs for accessing the Documentum features and managing the import and export of documents through Documentum. These APIs are increasingly being exposed through Java.
Java has been used for some additional customization of Documentum and for the agents controlling the rendering of the XML to PDF for print, HTML for the Web, and XML for export to 3rd party publishers. Java is an international standard, multi-platform language with the support of large vendors such as IBM, Oracle, and Sun Microsystems, and the open source community.
Borland's JBuilder has been used as the primary development environment for the Java code although any alternative Java development environment could easily be substituted.
Documentum provide a number of APIs specific to Documentum, and also generic XML APIs (DOM, call outs to XML Schema validators, XPath evaluators, XSLT engines, etc). Documentum version 5i leverages more Java than in version 4i.
XSLT has been used for the transformations from XML to HTML for the web. XSLT is the W3C standard for transforming XML and numerous engines are available to support the language including commercial and open source implementations.
A CSS stylesheet is made available by the PAL website to the user's browser in order to render the HTML page. CSS is the W3C standard for rendering structured documents including HTML5 and XML6 where complex reordering or multiple use of content is not required and numerous engines are available including good support in most web browsers.
C# has been used to implement the website application logic. C# is an international standard language championed by Microsoft as part of their .NET initiative (principally designed to compete with Java). While no other major vendors appear to have embraced it as a standard platform, Microsoft have indicated that .NET with C# as the primary language in that framework is central to their future direction and Microsoft are unlikely to go away within the lifetime of the PAL system. There are some open source initiatives to implement C# compilers and the .NET framework on other operating system and hardware platforms.
ASP.NET has been used to provide the framework of the Website implementation and is used as a wrapper around the C# code and the HTML generated from the XSLT transformations. ASP.NET is only supported within Microsoft's operating systems and web platform, IIS.
As part of the PAL solution, the Office of the Clerk has been provided with Word templates to facilitate its preparation of commentaries. These templates capture some existing macros and provide some standard paragraph and character styles to facilitate easy exportation as XML. Word documents produced by OC using these templates will be converted to XML using the Word importation features of Epic Editor. This process will be managed by PPU.
5.4 Conclusions about the PAL architecture
The overall PAL system architecture follows quite closely the standard pattern of systems of this type. The components selected at the high end of the market—the authoring tool and the DMS—reflect the understanding of the importance of powerful drafting tools to the drafters and support staff, the positive experience that PCO has had in the past with document management solutions, and the complexity of the legislative processes that need to be managed and tracked by the PAL system. An efficient and well-managed back-end process is crucial to providing timely, high-quality service and these selections are entirely appropriate. The components selected at the mid-range or low end—the rendering engine and the web site architecture—seem a little out of place given the capability of the other components. Because of the focus on public access of this project—high-lighting the newly acquired role of the PCO as publisher of legislation and related information—InQuirion has recommended considering upgrading both the print rendering and the website infrastructure.
With respect to the development environments, although a large number of languages and environments have been chosen, none of these environments is of particular concern from a maintenance point of view (other than the sheer number). By selecting a Java-based website solution and using Java for the Authoring Tool customizations and Documentum lifecycle customizations, the use of C#, ACL and VB and similar languages might have been eliminated but most programmers are used to switching from one language to another according to the suitability of the language to the task at hand. All of the languages chosen are suitable for the tasks chosen.
6. Specific questions
6.1 Is the application architecture implemented in a consistent, logical and understandable manner?
Section 5.1 above describes the typical architecture associated with an XML or SGML legislative drafting, management, and publication system. While New Zealand has selected a different tool combination from other jurisdictions, all of the pieces fit into the standard category of tools with some more advanced tools and some less advanced tools reflecting the different emphases, priorities, and experiences of the New Zealand environment. InQuirion is unaware of any other jurisdiction utilizing Documentum for XML legislation management. Perhaps this is because Documentum 4i was the first version of Documentum to support XML documents as anything more than another document type and provide chunking and XML metadata extraction. Most of the tool selections in these other projects took place before that of New Zealand.
While InQuirion has some reservations (set out below) about the capability or flexibility of a few components, the overall architecture is sound and has been implemented in a consistent, logical, and understandable manner. The resulting system architecture is no more complex than one would expect given the complexity of the domain and requirements.
6.2 Does the Epic software (Epic Editor, Print Composer, and E3) have the capability to meet PCO's requirements?
Note: Particular attention should be paid to the extent of stylesheet development in Phase One (pre-go-live) of the project, and the amount of further development planned for Phase Two (post-go-live to completion of the overall project).
6.2.1 Epic Editor—the Authoring Tool
Epic Editor appears to be a good fit to the PCO's requirements. In most jurisdictions where drafters have switched from a word-processor drafting environment to a structured editing environment such as Epic Editor (including Canada where they are migrating from Corel WordPerfect to ArborText Epic Editor), the reactions have been quite negative and drafters have required considerable convincing to migrate to the new environment. By contrast, the reaction of the New Zealand drafters both from PCO and IRD has been predominantly positive. A number of drafters have requested to use the new environment ahead of the system deployment! This openness and acceptance of the Authoring Tool is partly due to the provision of a number of extremely helpful, legislation- and DTD-specific features in the customizations of Epic Editor (worthwhile additions despite additional time and cost if only for this reason), and partly due to the drafters' familiarity and agreement with the aims of the PAL project and their understanding of their role not just as authors of legislative drafts but also as custodians of the statute book.
InQuirion notes that the drafters, secretaries, and PPU staff have generally expressed a preference to operate the Epic Editor in "tags visible" mode. This allows the user to position the cursor with greater accuracy ensuring that any content created is being placed within the right markup. With all the other windows and menus provided by Epic Editor open, and because of the density of the tags as specified by the DTD, this results in a fairly small amount of text visible as the user is drafting. User studies have shown that knowledge workers (of which legislative drafters are an example) generally perform more productively when they have more context available to them. In particular larger screen sizes and resolutions tend to make them more productive. An effective way to overcome this limitation would be to purchase large, high-quality monitors for the PCO staff.
There are some issues to do with management of non-English characters (Unicode) detailed in section 6.5.1.2, that can and should be addressed in the configuration of Epic Editor, but ArborText's documentation suggests that this is a relatively minor change.
6.2.2 Epic Print Composer—the Print Rendering Tool
The ArborText rendering engine, whether in Print Composer or E3, is a mid-range rendering engine. Its cost and its capabilities are lower than the high-end systems such as XyVision's XPP or Advent Publishing's 3B2 but higher than lower-end systems such as the Open Source FOP or a Microsoft Word-based solution. The initial evaluation included the high-end engine XPP from XyVision as well as the selected Print Composer and a number of other alternatives, but not Advent's 3B2. When InQuirion heard that Print Composer had been selected, based on viewing New Zealand Acts and Regulations, this seemed like a perfectly reasonable decision. The rendering requirements for consolidated Acts and Regulations were not particularly taxing, and, providing that Print Composer or E3 could satisfy the throughput and turnaround requirements of the PCO, it seemed that the increased costs for a high-end rendering engine were not really justified.
The ability to produce accurate, high quality, timely print renditions of legislation—in draft, final, and consolidated form—is crucial to the success of the PAL system. A number of different stakeholders have considerable reservations about the current print rendering solution. InQuirion shares their concern.
These issues focus around performance, functionality, and future fit, and the economics of continued deployment.
The rendering engine performance, the time taken to render large documents, is of considerable concern for a number of the stakeholders. InQuirion's interaction with the system suggested that rendering a single large document could take well over 20 minutes. Regardless of the lack of specific expectations in the formal requirements documents, this is clearly unacceptable by any reasonable subjective analysis.
In the current transform environment, only one such rendering can be done at a time so even small jobs might be delayed by more than 20 minutes if somebody else is working on a large Bill. Even with print previews (which are not bottlenecked like the transforms), most Authoring Tool users (whether drafters, secretaries, RU, or PPU) are likely to want to print preview a document many times a day. That performance will severely affect productivity if any user is working on a large Bill, Act, or Regulation.
Once the Parliament or a Committee has approved a document, there are often tight time frames in preparing it for the next stage, be it Assent or second or third reading. Minor changes early in a large document will necessitate repagination of the entire document. Any manually inserted page breaks will require user interaction. In order to set these breaks, a large document may need to be rendered many times, each time setting a new break and checking the resulting flow for subsequent pages. Therefore, what can be considered an acceptable rendering time is dependent on how the user invokes the stylesheet. A once off invocation can be significantly slower than an interactive invocation and still be usable. But Unisys is proposing that some issues be dealt with using manual intervention.
The bottleneck for transforms may be addressed by duplicating the print server (probably requiring the purchase of additional Print Composer licences) or by migrating to Epic E3, which allows multiple simultaneous rendering jobs, unlike the Epic Print Composer/Java agent solution that serializes all print requests. However, neither PCO nor InQuirion have been shown any evidence to suggest that individual print previews will be any faster (or indeed batch processes).
For reprints and bound volumes, the issue may be even more difficult. Since bound volumes number pages from the beginning of the first Act or Regulation in the year and end with the last page of the last Act or Regulation in the year, and Epic Print Composer (and InQuirion presumes E3) cannot set an initial page number, the entire year's collection will need to be rendered as a single document. How long is it going to take to render many thousands of pages and what other jobs will be blocked while this is happening?
It is possible that expanding the entity references in the FOSIs so that a single file is loaded rather than the large numbers of files currently loaded for each FOSI could reduce the rendering time for smaller documents. This is unlikely to be a significant factor in larger documents.
6.2.2.2 Functionality and future fit
InQuirion also has a number of concerns about the capability of FOSIs as implemented in the ArborText products as being sufficient to cope with the PCO's requirements. PCO's experience in the past has been that, when an issue is raised in one stylesheet, it is fixed there but the changes are frequently not propagated to other stylesheets or even similar elements in the same stylesheet. Frequently fixes to one aspect of the rendering have introduced new problems in other areas.
While it could be argued this is simply due to insufficient regression-testing infrastructure, InQuirion believes that it is symptomatic of the deeper issues discussed below.
Revision tracking
Although InQuirion has considerable experience with legislation and legislative drafting environments in a number of jurisdictions, every jurisdiction has its own unique requirements. New Zealand's extensive use of select committees (nearly every Bill is referred to a select committee) and committee of the whole House has lead to a sophisticated markup scheme to track the changes from first reading through the various committee processes to the third reading. The current system uses strike-through and underline for revision-tracked versions provided to the select committee, and subsequent changes are tracked using a combination of vertical and horizontal bracketing and other typographic markers.
As part of the PAL project, it was decided to use a modified form of the strike-through and underline style to represent changes coming out of select committees and the committee of the whole House for the consideration of the next legislative stage. As part of this decision, the level of changes was to be restricted to 2 (formerly up to 5 levels of markup were represented). These changes were made partly as a concession to the technology and partly to aid readability. This uses a mixture of single strikethrough and underline (to represent the first level of changes) and double strikethrough and underline (to represent the second level). For select committee output, bold strikethrough or underline represents a unanimous decision and normal weight strikethrough or underline represents a majority decision. Since ArborText's Epic Print Composer was unable to change the weights of strikethrough or underline independently of the weight of the font, a number of additional fonts were created to simulate this capability.
Although there are still outstanding issues with rendering revision tracking, this solution should be able to support the current PAL requirements for revision tracking.
The number of rules
One consequence of the revision tracking requirements is the very large number of rules. FOSIs typically contain many rules for each element, particularly if the element can appear in multiple contexts. ArborText products place limitations on the number of rules based on attribute values per element. This is because each rule has to support a single value for each attribute on each element. That means that an element with multiple values has to have a rule for each combination of different values of attributes. This can produce a huge volume of rules in some instances.
The volume of rules and the need to maintain identical, or worse, similar, rules in multiple stylesheets increases the development and maintenance overhead, the time taken for testing (as tests need to be applied to every different context), and the chance of human error in propagating changes in one stylesheet to all appropriate stylesheets and checking that new rules do not introduce unintended side-effects. While entity references have been used extensively in the FOSIs in an attempt to eliminate duplication, there are still numerous examples of very similar rules for an element appearing in multiple stylesheets. Most of the time, if one is altered, they all need to be altered in a similar way. ADG described a manual checklist that they use to propagate changes to 16 different contexts.
DTD issues
Legislation DTDs are necessarily complex. It is rare to have fewer than 100 elements in a legislation DTD. The markup is dense, typically 5-10 times more tags per word than normal prose. Even in a very carefully crafted legislation DTD, it is likely that the stylesheets will be large and complex.
But the PAL DTDs exacerbate this problem. The "Para" element appears in just about every possible context within the "Body" (and many others besides). However, most of the "Para" tags could be removed from the instances without affecting the representation of the logical structure of the document. While most tags in the PAL DTDs correspond to a concept or concepts that drafters have named (and use in reference and amendment wording), there is no such concept matching the "Para" tag. Ontario, Canada, Tasmania, PNG, and US Congress all have DTDs that are similar in structure to the PAL DTD but without the "Para" element. They support a wide variety of formatting and flexible editor interactions (including in some cases promote and demote). While New Zealand's revision tracking requirements are more extensive than these other jurisdictions, the revision-tracking markup in the New Zealand DTD is based on that in the Tasmanian and PNG DTDs. The "Para" element is completely independent of revision tracking.
Provisions within the body have a very regular structure (with a very regular formatting to match), but, within Schedules, this structure is much less rigid and the formatting is much more varied. The PAL DTD uses the same tag for provisions within the body as for provisions within Schedules.
These two factors are examples of how the existing DTD has made the rendering stylesheets more complex than they need to be. At this stage of the project, changing such fundamental parts of the DTD is not feasible. For instance eliminating the "Para" element would require changes to most Authoring Tool customizations, changes to the stylesheets for authoring, print rendering and transformation for the web. It would also require reconfiguration of the CMS and possibly changes to some of the Java code. Such a mammoth undertaking is not justified, as the benefits could be small and the costs in time and money considerable.
Large number of formatting issues outstanding
There are still a large number of formatting issues outstanding. A few minor formatting glitches are to be expected in introducing a new automated rendering system and some minor degradations in quality, such as unfortunate line or page breaks, are normal sacrifices in order to gain faster turn around. But legislative documents are amongst the most important documents in the land. The sheer number of formatting issues raises concerns about compromising the authority and status of the documents. Even if the issues can be easily overcome, a vigilant Opposition with the desire to delay or prevent the passage of some Bill, will not miss an opportunity to slow the government's legislative agenda by drawing attention to even minor presentation issues.
While the number of outstanding issues would be more easily resolved if there were one stylesheet not several, it is InQuirion's understanding that limitations of the rendering engine require the stylesheets to be split.
Line numbering
Among the unresolved formatting issues, there are still serious problems with the automatic line numbering of Bills. Any problem with line numbering (or any other rendering) is likely to distract the Members from the debate at hand. Committee amendments are worded using these line numbers and incorrect line numbers, particularly if two lines effectively receive the same number, will interfere with debate on the proposed amendment. PCO has currently been delivered version 4.3.1b of Epic Editor/PrintComposer. Fixes for line numbering have been included in version 4.3.1d but these fixes have not been delivered to, or tested by, PCO.
Manual intervention
Manual intervention is currently required to avoid a number of issues with the rendering. At least one formatting issue must be corrected by manually inserting page breaks in the markup to avoid the occurrence of a particular combination of elements at the top of a page.
The Epic suite also has the capability to manually alter the rendition directly (Epic Editor's Touch-up Tool). This results in processing instructions being inserted into the markup. While this is a standard industry approach where the problem is application-specific, if the need for intervention is inherent in the document instance rather than being application-specific, the changes should be incorporated into the element and attribute markup.
The danger of this approach is that, unless the XML with the processing instructions is correctly inserted into the repository, although a correct PDF is produced, the next time that XML document is rendered the old issues will reappear. Even if the processing instructions make it into the versioned repository, using such processing instructions limits the mobility of the data between applications. The lifetime of a legislative document may be many hundreds of years. The lifetime of a typical information technology system is rarely more than 10 years. Other rendering processes (in particular the generation of HTML for the web) currently ignore these processing instructions so these changes are not reflected on the web site. If these changes are simply changing a font size (which web users can do in their browser anyway), this is not problematic, but if it involves correcting autotext or correcting indentation for sandwich text, the changes may have significant semantic connotations and must be propagated to all publishing formats.
The importance of ensuring that the PDF created for the Parliament is the same as that sent to the website should not be underestimated. Currently, every time a document is promoted as part of the lifecycle, a new PDF rendition is created and, for those stages that involve tabling the document, sent to the website. This is currently done without checking for a previous PDF rendition whether created with or without manual intervention. At the very least, the CMS should check for a PDF document that has already been created and only create one if a previous rendition does not exist. This will prevent the possibility that the PDF sent to Parliament and to the web are different.
This approach of manually correcting rendering is part of the tradition of typography and publishing, but it fundamentally undermines one of the major drivers for utilizing structured markup such as XML. A successful XML system should allow the content to be rendered automatically without human intervention to each of a number of output formats—in this case PDF for a Bill, PDF for the subsequent Act if any, PDF for a later consolidation or official reprint, and HTML of any or all of these. This should be possible by matching the complexity of the rendering requirements with the capabilities of the rendering tool.
6.2.2.3 Economics of continued deployment
To address these and other issues, there is considerable stylesheet work proposed for both Phase 1 and Phase 2. This work combined with the editor customization is currently on the critical path for the Phase 1 deployment. Although review of the project plans and contractual arrangements is not within the scope of this review, the cost of this work alone, without factoring in the down-stream maintenance costs, is substantial.
Unisys suggests that Epic E3, if used as a replacement for Epic Print Composer, may address the concerns that InQuirion has raised. In order to replace the current rendering engine, Unisys has identified a number of sources of additional cost including:
- evaluation of the alternative,
- acquiring the new software,
- constructing the stylesheets,
- new integration work,
- project management,
- specification workshops,7
- doing the remaining work on the Authoring Tool, and
- additional training costs.
Of these additional costs, using Epic E3 over another alternative rendering engine such as XyVision's XPP or Advent Publishing's 3B2, potentially saves only the additional stylesheet work (since the same FOSIs as used in Epic Print Composer apply) and some of the additional training (since the FOSI training already received presumably still applies).
While Unisys claims that much of the stylesheet work proposed for Phase 2 is for the Authoring Tool environment rather than print rendering (specifically around Reprints and Bound Volumes), there are a number of level 1 and 2 issues required to be fixed before deployment of Phase 1 and a number of level 3 and 4 issues that are currently scheduled for Phase 2. This is in addition to the reprint and bound volume work (which requires both editor and print rendering customization). The argument in favour of using ArborText technologies for both authoring and print rendering relies on being able to share that work. However, if the print rendering is still not satisfactory, a new replacement solution may still be required. The economic argument of shared work between the authoring tool and the print-rendering tool loses much of its appeal.
The complexity of the FOSI rules and the reappearance of old issues in new contexts in past work suggests that work on reprints and bound volumes is just as likely to create new issues as to resolve them. Higher-end tools such as XPP and 3B2 use more flexible rule models than FOSI, and the effort to create the stylesheets, and the effort for ongoing maintenance of the stylesheets could be considerably reduced. While these tools use proprietary stylesheet languages, not many vendors support the FOSI standard so the benefits of an open standard are limited and tend to be outweighed by its limitations.
XPP was evaluated in the initial rendering engine selection process but because of the high cost of the product and the lack of a local distributor, 3B2 was not considered. Since the evaluation, Advent Publishing has appointed an Australasian distributor (Allette Systems based in Sydney) and is aggressively pursuing the Australasian market with local support and significantly reduced licence fees. This reduces the costs and risks of development and ongoing maintenance of a solution based on 3B2.8
While it was not within the scope of this review to examine project plans and cost and schedule implications, InQuirion's experience with the 3B2 rendering engine as used in the Canadian legislation project with ArborText's Epic Editor suggests that it is possible that the costs of acquiring and configuring 3B2 could be lower than the costs of addressing the current issues with the ArborText products and the development time frame should not be adversely affected. Further cost savings in lower maintenance and ongoing development costs are also likely. The same is possibly true of the XPP engine from XyVision (although InQuirion's familiarity with this engine is not as great).
It has been claimed that ArborText have never reached the attribute limitation in any other implementations despite implementing a number of legislation solutions. This suggests to InQuirion that perhaps the ArborText tools are being used beyond their capabilities, either because the New Zealand requirements are more complex than other jurisdictions (which is certainly true of the revision tracking requirements), because the DTD is more complex than other jurisdictions' DTDs (through a mixture of requirements and design decisions), or because the stylesheets have not properly utilized the product's capabilities.
Unisys subcontracted out development of the document analysis and DTD design to eGloo and the FOSIs and other Authoring Tool customization to Absolute Data Group (ADG). The eGloo principals have worked on a number of legislation-related projects including the NSW PCO. While there are some issues with the DTDs as described above, the general philosophy behind the DTD design is sound. Addressing these issues is likely to create large amounts of additional work without any guarantees of significant change in complexity of the FOSIs or improvements in reliability or throughput. ADG are trained in the ArborText products and experienced in developing FOSIs and editor customizations with ArborText and other products. InQuirion has not identified any major issues or inconsistencies in their customization. This suggests that New Zealand's requirements possibly justify a more flexible rendering engine.
For these reasons, InQuirion has strong reservations about the ability of Print Composer to support the current rendering needs of the PCO, let alone the future needs. InQuirion is happy to entertain that E3 may be able to satisfy PCO's requirements. Our concern is that, while E3 may address some or all of the performance issues, the complexity and maintenance overheads of FOSIs will remain along with the outstanding functional and formatting issues.
In order to address the issues of specification which have been raised as problematic by Unisys, InQuirion recommends that PCO should prepare a set of sample documents, the rendering specification, and example PDF or paper output showing what PCO consider the output should look like (see section 7.2) together with some indication of what issues PCO feel are most problematic in the current environment (e.g. throughput, line numbering, revision tracking).
This should address the issue of specification workshops without additional cost to Unisys. This test set is necessary for acceptance testing, but could also be used for unit, system, and integration testing. However, InQuirion recommends that the test set be provided to a selected number of vendors to demonstrate their capabilities with those examples on candidate replacements for Print Composer including ADG (ArborText) with E3, Allette (Advent Publishing) with 3B2, and possibly XyVision with XPP. PCO and Unisys should seek a fixed price quote from subcontractors based on this example set so it is in PCO's interest to ensure complete coverage of all of the issues.
While InQuirion acknowledges that the risks of switching rendering engines at this late stage of the project are not trivial, we consider that the risks of retaining the existing rendering engine and customizations are comparable.
6.3 Does Documentum 4i have the capability to meet PCO's requirements?
Note: Particular attention should be paid to chunking.
The current release of the PAL system utilizes Documentum's version 4.3.2 client and 4.2.3d server. InQuirion believes that PCO could go live with these versions, however it may be prudent to consider upgrading to 5i earlier rather than later (5.2 seems the most appropriate candidate). The PCO and IRD require the ability to chunk Bills at Parts and Subparts so that multiple drafters can work on different Parts or Subparts simultaneously to increase the throughput for drafting large, complex Bills (see Content Management System Design section 7). The main issues are:
- 1. Documentum 4.3 does not support the ability to turn-off chunking just for those Parts and Subparts that appear in amendment wording (as provisions to be inserted) without inserting additional attributes, which would require additional work in the Authoring Tool and add more application-specific markup to the XML data;
- 2. Documentum 4.3 does not provide a mechanism for autotext to be used in the metadata describing the chunk, meaning that there is no simple way to display autotext Part numbers when browsing through the list of chunks that make up a particular Bill (once the Part number is fixed, this problem vanishes);
- 3. Documentum 4.3 can be significantly slower at checking in large XML documents with large numbers of chunks than the current release; and
- 4. Documentum 4.3 relies on four attributes that it inserts into the elements on which it chunks to identify the chunk when later versions are checked in. If in the editing environment, users naïvely copy and paste an element that is a chunk, these attributes are reproduced even if the vast majority of the content is changed. This is actually a feature rather than a limitation of the product but it is not the desired behaviour in a legislative drafting context and can only be addressed by Authoring Tool modifications. A package has been created by ADG for the Authoring Tool but it does not replace the existing cut-and-paste functionality. It is merely an additional tool. ArborText claim in their release notes that this is fixed in ArborText Epic Editor 4.4 for the Documentum/XML Adapter.
With the exception of issue 4, these chunking limitations are potentially annoying for the users but do not affect the data integrity and issue 4 cannot really be addressed in the CMS.
Documentum have assured InQuirion that Documentum 5 has addressed issue 1 allowing fragmentation rules to refer to their parent elements without causing exceptions. They have also spent considerable effort optimizing the XML capabilities of the product (including fragmentation) and suggest that considerable performance improvements can be gained by upgrading, therefore addressing issue 3. They have also suggested that upgrading from 4 to 5 should require very little if any coding changes. Going live with a release that is already more than a year old (4.3) means that PCO is likely to have to upgrade within a year of going live anyway (Documentum typically support a release for about 2 years after releasing). PCO already have licences to the upgrades included in their software maintenance agreement. Given that the CMS is not on the critical path for either of the Stage 2 phases as currently planned, despite the belief that 4.3 is an acceptable release on which to go into production, the benefits of upgrading earlier are compelling.
Note that, in order to receive the performance and functionality benefits, both the client and the server installations would need to be upgraded.
6.4 Does the mix of package customisation and bespoke development support future development and package upgrade without major rewrites or design changes?
Note: Particular attention should be paid to the amount of development required for Phase Two (post-go-live) of the PAL project, and the ability to further develop the system beyond Phase Two.
It is inevitable in a large IT project such as this one that no one package will do everything that the users require "out-of-the-box". The PCO sought an Implementation Partner because they recognized a need to configure and customize the generic applications (Epic Editor, Epic Print Composer/E3, Documentum) to support the business processes and operational requirements of the users.
Such configuration and customization always risks that future versions of the software on which the system is based will require changes to that customization. The extent to which changes are required is largely in the hands of the vendors of the underlying software components. Comments in this section rely heavily on information supplied to InQuirion by ArborText and Documentum. Some speculation is also included based on InQuirion's knowledge of the broader XML market.
The lifetime of such a large IT system is usually of the order of 5- 10 years and predicting anything in the IT market beyond a year or two is highly speculative at best. An unequivocal answer to this question is therefore not prudent or practicable. InQuirion has identified a number of issues pertinent to future development and package upgrade for each of the components of the PAL system.
The Authoring Tool configuration uses FOSIs for rendering the documents and ACL packages to implement custom tools for authors. These configurations and customizations cannot be readily migrated to alternative authoring tools from other vendors. In the unlikely event that PCO would choose to migrate to an alternative authoring tool in the near term, new stylesheets and customization would probably need to be developed from scratch for the new target authoring platform. But ArborText has supported both FOSI and ACL for many years and has a large installed user-base with customizations using these tools. They are unlikely to discontinue support for either in the near future. They also have released a tool that reads in existing FOSIs, allows modifications to be made, and outputs either FOSIs or XSL-FO based transformations. If they were to discontinue FOSIs in the future (presumably in favour of XSL-FO), this tool should minimize the human effort required to migrate. Migration from one version to the next of Epic Editor in the past has required little or no additional coding. ArborText also have an excellent record in supporting older versions of their product.
Looking ahead a little further, Microsoft has released beta versions of the next version of Word (2nd quarter 2003), which provides an XML editing solution as part of its Professional Office suite. While this product is not as mature or sophisticated as Epic Editor, it is likely to have an impact on the market for Epic Editor. In the short term, this may actually increase the market as XML solutions become widespread but power users require more than the new version of Word offers. As future versions of Word match the capabilities in Epic Editor, the long-term viability of the product may be in jeopardy, but this is likely to be many years into the future, probably beyond the useful life of the current PAL implementation. If it were to dominate earlier, Documentum are already working on integrations to the new XML capabilities so migrating to Word as an XML authoring platform would not have a significant impact on the CMS.
InQuirion's concerns about the rendering engine, Epic Print Composer, are explored above in section 6.2.2. The major concern with future development is the complexity of the FOSI stylesheets and the maintenance burden of managing so many similar stylesheets.
InQuirion agrees that maintenance of stylesheets would be much simpler if the number of stylesheets were reduced but understands that the number of FOSIs was increased partly because of limitations of Print Composer. Tasmania and PNG both have a single print rendering stylesheet that covers Bills at every stage, Acts for assent, for loose- leaf publication, annual volumes, and consolidations as well as draft, final and consolidated regulations.
Looking more long term, CSS has effectively replaced FOSI as a language for describing how to display XML documents, and XSL-FO is emerging as the standard page description language in the XML community. While XSL-FO still has a number of limitations that prevent it from being used in high-end publishing environments, these limitations are certain to be addressed within the next few years. ArborText, as a leader within the XML community, has already embraced XSL-FO and the rendering tools used in the PAL project support using XSL-FO to render XML to paper as well as the FOSI approach adopted. At some point in the future, it will become feasible to migrate to an XSL-FO solution, which would give the PCO more freedom to change rendering engines. InQuirion supports the conclusions in Authoring Tool End User Specification that FOSIs currently provide superior formatting capabilities to XSL-FO and, given the selected rendering technologies, were t