How to prequalify vendors
Identify your candidates and send them the RFP. Ask them questions which will separate the wheat from the chaff, for example:
* Please present the team which will carry out the project. How much experience do they have with Scrum and XP (Extreme Programming)?
* Are you willing to organize the project according to Scrum (as described in the Process chapter)? What experience do you have with agile projects on this scale?
* Please estimate the user stories in story points. What is your estimate of overall complexity (size) of the system in Story Points?
* What is your expected team velocity in Story Points per Sprint?
* Given our target budget, how much of the functionality do you think can be realized?
* Given the ground rules, are you willing to participate in the competition to select the final vendor?
How to prequalify vendors
Identify your candidates and send them the RFP. Ask them questions which will separate the wheat from the chaff, for example:
Hedge your bets on productivity differences between teams
Productivity differences between individual developers can be a factor of 10 to 20. Teams converting to Scrum often report a 3 fold increase in productivity within 6 months. The best Scrum teams have reported improvements of a factor of 10 compare to industry averages. The difference might be technical ability, but human chemistry issues are just as important, if not more so.
Even if the difference between to the top two contenders is only 25%, investing 10 to 20% of the development budget to hedge your bets reduces your risk substantially and may pay off dramatically.
Let us assume that you plan to spend $2.4 Million over 12 months, or $200'000 / Month to develop the software. If you add one month's effort to the budget, that would raise the total by 8%. But given the productivity differences between teams, even investing an additional 25% probably yields a positive ROI. Furthermore, the cost of delay while you take three months to pick a partner without producing any usable software should be much larger than the cost of redundant development for a short period.
To effectively procure the services of an agile development team, an "agile RFP" would require the supplier to:
* Indicate who are the people, or at least the type of people, that the supplier will have on the project. The supplier should provide resumes and the customer will likely choose to interview some or all of the team.
* Follow a reasonable set of rights and responsibilities for both the customer and the supplier to adhere to throughout the project. These rights and responsibilities will provide the foundation for how they will collaborate together.
* Propose how they will work with the customer organization.
* Produce potentially shippable software to the customer every X weeks (for me, X is usually 4 or less).
* Allow the customer to monitor their work and work products. Depending on the situation, the customer may insist on having one or more of their people on the supplier's site throughout the project. The customer should insist on access to the source code at all points in time so that they can inspect it at their leisure, including but not limited to running it through a static code analysis tool. They should also insist on access to a project management dashboard whose metrics are populated automatically, in (near) real time by the tools the supplier's development team is using.
* Follow the customer's corporate development guidelines which they will provide to the supplier. The customer will be monitoring their compliance to these guidelines throughout the project.
* Do full regression testing throughout the lifecycle. The customer should favor any bid which indicates that the supplier will take a test-driven development (TDD) approach.
* Provide a minimal set of deliverable documentation, particularly user documentation, operations documentation, support documentation, and high-level systems overview documentation. The customer should provide examples of such documentation if they're available.
* Accept a realistic payment strategy that is based on a low time