How to Evaluate a Vendor

Unless you’ve done a full discovery process, I never trusted the generic vendor-provided survey/ROI reports. One of the ways that I’ve found to really learn about a product is to try to get enough funding to take one of the vendor’s low-level training classes. That’s a lot cheaper than buying a product and then finding out it didn’t hold up (why does any company think it’s a good idea to just buy and find out?). The training class method is good, because, when they’re training you on a product, you usually have someone who knows the product inside and out. This gives you a direct channel to ask better questions and get the real answers – instead of the sales-guy spiel. Might be an option?

Gathering requirements is a difficult task, but it’s something that we do every day. There are always some general requirements that fit with nearly every single-sourcing project.

These kinds of requirements break up into the following categories:: General System Requirements; Requirements for all applications and vendors; Additional XML Authoring Tool Requirements; Additional Workflow Tool Requirements; Performance Requirements; Data Object Requirements; Network Requirements; Benchmarks for Requirements Verification

These are the underlying requirements that define any good system: the ability to control document source, the ability to create and produce documentation products for customers. The goal of a good requirements document is to capture and define requirements for a new system above and beyond these basic requirements. A good requirements document includes these basic, best practices requirements, in addition to the problem-specific requirements that can only be defined in terms of your business processes and needs.

These are the kinds of requirements will help to ensure a system that encourcages user participation. It helps you to design a system that can provide real benefits to the user and to the company. A well-designed system will provide many opportunities for small efficiencies in addition to any benefits due strictly to upgrading to new technology.

It is my intention here, then, to lay out some best practice requirements, to get requirements documents writers started. This is a jumping off point. Not an end point. The real end point includes all those intangibles that only you know about as the in-house expert, the requirements document writer.

General System Requirements: Can keep data local to task; Minimizes overhead; Fits into existing business practices; Optimizes, improves, or otherwise streamlines existing business practices; Does things right; Does not compromise customer-driven or business-driven requirments; Avoids unnecessary structure for the sake of structure; Can customize metadata/structure to reflect real business requirements; Guarantees functional quality with required output systems; Guarantees informational integrity; Improves quality; Improves accuracy; Improves flexibility; Guarantees consistent presentation; Supports authoring; Supports editing; Supports document management/configuration management; Supports workflow/tracking; Supports production

Requirements for all applications and vendors: Is currently supported; Uses current technology; Has plan for incorporating new technology; Has clear path for upgrades; Has clear plan for product deliveries; Is customizable; Is extensible; Has clear, defined plan for upgrading customization code; Supports automation through customization or extension; Application itself can be automated; Designed to scale to meet new customers and future company requirements

Additional XML Authoring Tool Requirements: Compatible with graphics output (e.g., SVG, CGM, EPS, TIFF, GIF, JPG, PNG, BMP) output; Compatible with graphics input (may be same as or a subset of those formats listed above); Vendor has lifecycle policy; Produces documents that are of comparable quality; Supports content reuse; Supports custom-defined chunking; Supports smaller than single-document chunking

Additional Workflow Tool Requirements: Has control mechanisms; Has approval mechanisms; Has guaranteed audit trail; Records transactional historical data; Tracks user tasks and events; Records: state transistions, approvals, times/dates, routing activities, user; Can automate the creation of ancilliary documents; Can interact with other external systems; Can fire off external events; Can be advanced through external mechanisms (e.g., API); Can store additional metadata about stored data objects; Uses Lock-Modfiy-Unlock or Copy-Modify-Merge (Specify which); Can define roles; Can define access requirements; Can use role-based access to guarantee entitlement; Integrates with LDAP; Supports execution of custom, exernal, automation tools

Performance Requirements: Side-by-Side Performance Test; Check-in/Check-out Directory; Check-in/Check-out Single-File; Check-out of documents for edit; Check-out of documents for read-only; Level 1 Workflow testing (edit, route for review, annotated, dispatch); Level 2 Workflow testing (Level 1 + re-edit, 2nd draft, route, dispatch, approval); Level 3 Workflow testing (include full production run); Testing with multiple clients/multiple users; Tangible improved level of functionality achieved; Tangible improved level of performance achieved; Tangible improved level of integration achieved

Data Object Requirements: Binary Document types (MS Office, FrameMaker, PageMaker, Interleaf, etc); SGML Files; XML Files; Metadata extraction from SGML/XML file (CMS integration); XML/SGML file internal link management; Binary graphics file type: EPS; Binary graphics file type: TIFF; Binary graphics file type: GIF; Binary graphics file type: JPG; Binary graphics file type: BMP; Binary graphics file type: WMF; Binary graphics file type: CGM

Network Requirements: Directory structuring; Permission structuring; Roles-based permission entitlement

Benchmark Test Requirements: Task: Authoring Application Launch; Task: CMS Launch; Task: Check-out directory; Task: Check-in directory; Task: Create new document; Task: Delete document; Task: Grant user permissions/access; Task: Change user permissions/access; Task: Look at object properties/metadata; Task: Look at object history/audit trail; Task: Retrieve old version of document; Task: Check-out single-file-document; Task: Check-in single-file-document; Task: Search for document; Task: Change user ownership; Task: Generate object report; Task: Check-out SGML/XML-document; Task: Check-in SGML/XML-document; Task: View Tasks/Jobs; Task: Acknowledge/Claim assigned task; Task: Review Task; Task: Add Reviewer; Task: Approve; Task: Dispatch; Task: Create revision/tag/branch; Task: Create release