Stackrate

Methodology

Every evaluation is a rule, a source, and a citation.

Reports on Stackrate are not opinions. Each finding is produced by a documented pipeline, backed by cited sources, and graded by the type of source it relies on. This page shows you how — openly enough that a buyer can audit any number, and a vendor can see exactly what it takes to raise a grade.

For buyers

How to interpret what you see

Every vendor on a report gets two numbers, deliberately shown at equal weight. They answer different questions, and reading them together gives a more honest picture than any single score could.

01

Fit score — how well does this vendor match the buyer's needs?

A weighted percentage, 0–100%, computed from every finding in the report. Four inputs fold in:

  • Finding status

    Supported, partial, unclear, or not-supported.

  • Requirement priority

    Critical, important, or nice-to-have — critical weighs 3×.

  • Quantitative containment

    If the buyer asks for a number, does the vendor's proven envelope cover it?

  • Requirement dependencies

    A shortfall on an upstream capability cascades into everything downstream of it.

Verdict bands

Strong fit

80–100%

Good fit

65–79%

Moderate fit

50–64%

Significant gaps

0–49%

One hard rule. Any vendor whose proven envelope falls short of a criticalbuyer ask is capped at “Moderate fit” and labeled a critical miss — no matter how high the raw math would otherwise compute. A quantitative miss on a must-have cannot be papered over by strengths elsewhere.
02

Confidence grade — how verifiable is this assessment?

The confidence grade is derived from the type of source each finding relies on, not the content of the finding itself. A claim backed by a help-center article that names the mechanism is more verifiable than the same claim backed by a marketing page that only asserts the outcome.

The grading ladder

A

High confidence

Help center, technical documentation, or developer docs

B

Solid confidence

Product page with named mechanism (not just claim)

C

Low confidence

Marketing page, category hub, pricing, or homepage

D

Weak confidence

Blog, opinion, or positioning content

V

Vendor-verified

Verified vendor response with submitted evidence

Worked example

Example

Tipalti — 8 sources across 8 findings. Final grade: C · Low confidence.

Source typeGradeCountPoints
Help-center URL (support.tipalti.com)A11.00
Marketing / product pageC31.65
Blog / opinionD41.40
AverageC80.506
On the report, this reads: Tipalti 53% · Moderate fit · C · Low confidence. Two separate signals. The fit score says the vendor partly matches the need; the confidence grade says the assessment leans on marketing and is worth investigating further before deciding.
03

Fit × confidence — the four-quadrant reading

The two scores answer different questions and are deliberately shown at equal visual weight. Reading them together is what turns a report into a decision.

High fit×High confidence

Trust the recommendation.

The vendor matches the need and the assessment is well-evidenced. If you had to act, this is where you'd start.

High fit×Low confidence

Promising — investigate.

Findings point to a good match but lean on marketing. Ask the vendor for documentation or a technical demo before committing.

Low fit×High confidence

Firm rejection.

The gaps are real and well-documented. Move on unless something material changes.

Low fit×Low confidence

Unclear — don't decide yet.

We can't say whether this vendor fits. Ask for documentation, request a deeper evaluation, or skip.

04

Why ‘unclear’ is a valid answer

Most software evaluations force every finding into a binary: the feature is supported, or it isn't. That forces the evaluator to make things up when the evidence is thin. We don't. A finding whose evidence genuinely doesn't support a conclusion is marked unclear, and the report says so openly.

Treat “unclear” as actionable, not as a dodge: it means we looked and couldn't find a defensible answer. The vendor can fix it by submitting evidence, the buyer can fix it by asking the vendor directly. Either way, it's information.

For vendors

How you're assessed — and how to respond

If you're a vendor evaluated on Stackrate and a finding looks wrong, there is a structured way to respond — no support tickets, no lobbying. The grade is a function of the sources behind your findings, and sources can be added.

01

How to reach Grade A on a finding

The fastest path is almost always to submit a help-center URL. Most vendors can raise several findings to Grade A in under an hour.

  1. 1

    Claim your profile

    One form. The claim is tied to an email at your company domain, and unlocks the vendor portal for your listing.

  2. 2

    Open your report in the vendor portal

    Each finding shows its current grade and the sources behind it. Findings at C or D are flagged with a one-click “Raise this finding” action.

  3. 3

    Submit a help-center URL — or a verified response

    Our pipeline fetches your URL, confirms the mechanism is described there, and regrades the finding. If no public doc exists yet, submit a verified response with supporting evidence and our AI review cross-checks it against what you provided.

  4. 4

    Grade updates immediately

    Approved URLs move the finding to Grade A. Approved verified responses become Grade V, cited inline on the public report. Both update the vendor's overall confidence grade in real time.

02

What a response looks like

Help-center URL

Raises the finding to Grade A

When to use

Your mechanism is already documented publicly.

How it works

Paste the URL. Our pipeline fetches it, verifies the mechanism is described, and regrades.

Typical turnaround: Minutes

Verified vendor response

Raises the finding to Grade V, cited inline

When to use

No public doc exists, but you can describe the mechanism with supporting evidence.

How it works

Submit the response + supporting attachments. Our AI review cross-checks against what you provided.

Typical turnaround: Hours

Editorial dispute

Reviewed by Stackrate; all decisions logged publicly

When to use

You believe the rubric or a finding's logic is wrong, not just the sources.

How it works

Open a dispute against a specific prompt or scoring rule. Every accepted and rejected change is logged with the reason.

Typical turnaround: Days
03

The intentional tradeoff we're willing to defend

A vendor with a great product and a poor help center will look worse here than a vendor with a mediocre product and thorough docs. That outcome is intentional, because a buyer cannot verify a mechanism they cannot read about — and we'd rather give buyers a signal they can audit than a score that flatters vendors who didn't publish their documentation.

It's also the most fixable problem in this report. If you ship documentation, your grade rises. If we get something wrong in how we read your documentation, the editorial dispute path is open and the decision is logged publicly.

Editorial

Safeguards built into the methodology

A methodology is only as trustworthy as the rules that constrain it. These are the structural protections — the ones that hold regardless of who runs Stackrate or which vendors are being evaluated.

Every claim is cited

No finding is published without a source. The source itself is graded; the grade is published with the finding.

No vendor can pay to change a finding

Sponsorship, paid placement, and subscription tier do not influence findings. Verified responses are evaluated on their evidence, not on who submits them.

Prompt edits are versioned and logged

Every change to the AI pipeline is recorded. Reverted and rejected changes are part of the audit record, not just the ones we accepted.

Disputes are public

Accepted and rejected editorial disputes are published with the reason. If a vendor argues a rule is wrong and we agree, it's visible. If we disagree, that's visible too.

Unclear beats invented

When evidence genuinely doesn't support a conclusion, the finding is marked unclear. We don't force every call into a binary.

Grades are fixable

Every C and D finding has a defined, one-click path to a higher grade. A low grade is a statement about evidence, not a judgment locked in place.

What we don't measure

Intentional exclusions, with reasons

A methodology is defined by what it refuses to measure as much as by what it does. Each of these is a deliberate choice, not an oversight.

Pricing

Context-dependent, changes frequently, and poorly represents the real cost of ownership. We link to vendor pricing pages instead of republishing stale numbers.

Customer satisfaction, NPS, review aggregates

Not reproducible and not auditable at the mechanism level. Good sentiment signals exist elsewhere; our job is to tell you how the software actually works.

Brand reputation or analyst firm rankings

We measure mechanism fit for your specific scenario. Brand-level rankings are a different question, answered better by different sources.

Implementation partners, SI firms, training vendors

Adjacent to the evaluation but not part of the product. Noted when relevant; not part of the fit or confidence score.

Unverified user-submitted opinions

If a claim cannot be traced to a documentable source or a verified vendor response, it doesn't appear in a finding.

Where Stackrate sits

Compared to analysts and review sites

Stackrate isn't a replacement for every software-evaluation resource. It's designed for a specific job: mechanism-level fit for a buyer's specific scenario, with every claim auditable.

 StackrateAnalyst reportsReview sites
Primary signalMechanism fit for your scenarioAnalyst opinion, vendor relationshipsAggregate customer sentiment
Source transparencyEvery claim cited; every source graded A-DMethodology described; sources not publishedReviews visible; weighting opaque
Scenario-specificYes — you describe your process, report is built to itPartial (use-case-segmented reports)No — global aggregate
Updateable by the vendorYes — submit docs or verified responseNo (outside of briefings)Only by soliciting new reviews
Cost to buyerFree$25k+ subscriptionFree, with upsell

Depth

The full AI pipeline

Every report is produced by the pipeline below. The prompts here are the live prompts resolved against the override store the same way the runtime pipeline resolves them — nothing paraphrased, nothing simplified. Long on purpose.

Shared context blocks

Reusable pieces composed into multiple pipeline steps. Editing any one of them affects every step that composes it — which is why they are written, versioned, and reviewed separately from per-step instructions.

Shared block

Citation Rules

How to cite: web search is primary, training knowledge marked as such, never fabricate URLs, every factual claim needs a source.

The credibility guarantee of the product. Bad citations = no trust. These rules make Claude honest about where each claim came from.

Used by: 3. Find evidence and write findings

**SOURCE CITATION RULES**

You have two primary sources of knowledge:

1. **Web search results and official vendor documentation**: Use web search to find the specific help center articles, technical documentation, implementation guides, and support articles that address the buyer's requirement. Always prefer official vendor help centers over marketing pages. Cite the specific document or help article found.

2. **Your training knowledge**: You have extensive knowledge of vendor products from your training data. When citing from training knowledge, name the specific document or help article and mark it as "(from vendor documentation)" or "(from vendor blog/marketing; verify current availability)."

Rules:
- **Web search is your primary tool.** For each vendor and requirement, search the vendor's help center to find the SPECIFIC feature that addresses the buyer's need. Do not rely solely on training knowledge.
- Never fabricate a specific URL. Only cite URLs returned by web search.
- Every factual claim about a specific vendor capability must have a source attribution.
- If web search and your training knowledge both lack information about a capability, then and only then mark it as "unclear."

**WRITING STYLE**
- NEVER use em dashes (—) in any output. Use colons, commas, semicolons, or periods instead. The platform's editorial voice does not use em dashes under any circumstances, including inside citations, quotes, parentheticals, or bullet lists. If you would normally write "X — Y" write "X: Y" or "X. Y" or split the thought into two sentences.
- Do not use en dashes (–) as substitutes for em dashes; hyphens (-) are fine for compound words only (e.g., "3-way match").

Shared block

Status Definitions + End-to-End Fit

Strict definitions of supported / partial / unclear / not-supported, plus the end-to-end process fit test that every status must pass.

The most important rule in the system. Without this, Claude would rate anything that technically exists as "supported", even when the mechanism breaks the buyer's process downstream.

Used by: 3. Find evidence and write findings

**STATUS DEFINITIONS; apply these strictly based on evidence, not vendor reputation**
- "supported": The vendor handles this well at the depth the buyer needs. Mechanism is clear, no material shortfall for this specific requirement. You can assign "supported" based on your training knowledge; you do not need reference data to confirm it.
- "partial": The vendor offers a version of this capability, but it falls short in a material way; e.g., 2-way matching instead of 3-way, fixed approval chains instead of flexible stakeholder involvement, header-level OCR instead of line-item extraction, subset of ERP fields instead of full fidelity. Partial is not a hedge; it means the buyer will hit a ceiling.
- "unclear": You genuinely do not have enough information from ANY source (reference data, training knowledge of documentation, training knowledge of blogs/marketing) to assess this capability. Do NOT mark something as "unclear" simply because it is absent from the reference data provided in this prompt. Only use "unclear" when you truly don't know.
- "not-supported": The capability is absent or requires a third-party add-on not included in the base product.

**END-TO-END PROCESS FIT; this is mandatory for every status assignment**
The software categories Stackrate evaluates are multi-step process systems. A buyer's process description defines a chain of steps where data flows from one stage to the next. When you evaluate a requirement, you are evaluating one step in that chain; but the mechanism you identify must be compatible with every other step the buyer described.

Before assigning "supported" to any requirement, you must test: does the mechanism I am proposing for this step allow the buyer's upstream and downstream steps to work as described? If a vendor's mechanism for step 1 breaks or prevents step 3, that mechanism is "partial" at best; even if the mechanism itself is real and well-documented.

Example: A buyer needs invoices segregated by project during processing, then consolidated for centralized payment. A vendor that achieves segregation through separate legal entities technically provides data isolation; but it breaks the consolidation step, because separate entities mean separate books, separate payment runs, and no unified AP queue. The correct status is "partial" or "not-supported," not "supported," because the mechanism is incompatible with the buyer's end-to-end process.

The test is always: if the buyer implemented exactly the mechanism you are describing, would their full process; from the first step they described to the last; work end to end without restructuring their operations?

Pipeline steps

Each step takes structured input, runs a Claude call (or parallel per-vendor calls, for the findings step), and hands structured output to the next step. User-message templates use {{placeholder}} syntax for values filled in at runtime.

Step 1

Extract requirements

Reads the buyer's process description and returns 5-8 specific, testable requirements tied to their exact situation.

Role

Translate a narrative process description into a structured list of requirements the rest of the pipeline can evaluate.

Model

Claude Sonnet (claude-sonnet-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • Process description (free text from the buyer)
  • Optional source notes (e.g., uploaded RFP content)

Outputs

  • Array of requirements, each with description, category, priority (critical / important / nice-to-have)

User message

The instructions Claude reads when turning a process description into 5-8 testable requirements.

Step 2

Build search plan

For each requirement, decides what to search for on each vendor's help center, including anti-patterns that would break the buyer's process.

Role

Pre-analyze each requirement so the findings step searches for the right features with the right vendor-specific terminology, not generic terms.

Model

Claude Sonnet (claude-sonnet-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • The requirements extracted in step 1
  • The list of vendors being evaluated
  • The original process description

Outputs

  • A search plan per requirement: solution type, possible mechanisms, anti-patterns, and 3-5 targeted search queries per vendor

User message

Asks Claude to design vendor-specific search queries for one requirement before findings run.

Step 3

Find evidence and write findings

For each vendor × requirement: does 2-3 web searches, then writes a "how it works" finding with inline citations and a status.

Role

The core of the product. Produces the evidence-backed per-vendor per-requirement answer that the rest of the report is built on.

Model

Claude Sonnet (claude-sonnet-4-6)

Tools

web_search_20250305

Sources consulted

web-search, knowledge-base, vendor-comments, vendor-docs, search-plans

Shared blocks composed

Status + E2E fit, Source rules

Inputs

  • The requirements from step 1
  • The vendors to evaluate
  • The process description
  • Search plans from step 2
  • Knowledge base hints (topic and prior-found URLs per vendor)
  • Vendor-provided documentation (if uploaded by vendors)
  • Approved vendor comments on prior reports (first-class source)
  • Pre-fetched help-center content (from enrichWithWebSearch)

Outputs

  • ReportFinding per vendor per requirement: status, howItWorks, limitations, confidence, sources

User message

The core prompt that drives per-vendor per-requirement evaluation with web search + inline citations.

Step 4

Validate mechanisms (cross-requirement)

Checks whether the mechanism proposed for one requirement breaks another requirement. Only downgrades, never upgrades.

Role

The end-to-end process fit guarantee. Catches cases where a vendor technically supports every requirement individually but the mechanisms conflict end-to-end.

Model

Claude Opus (claude-opus-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • All findings from step 3
  • All requirements
  • The process description

Outputs

  • Updated findings, with some downgraded (supported → partial, etc.) when a mechanism conflict is detected
  • mechanismConflict metadata on any downgraded finding

User message

Cross-requirement compatibility check. Asks Claude to flag genuine structural conflicts for one vendor.

Step 5

Executive summary

3-5 sentences naming strongest and weakest vendors for the buyer's situation and calling out critical gaps in operational terms.

Role

The CFO-readable top of the report. Must name concrete support counts and specific operational impacts, not generic marketing language.

Model

Claude Opus (claude-opus-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • All requirements
  • All findings (post-validation)
  • Vendor list
  • Process description

Outputs

  • A 3-5 sentence executive summary string

System preamble

Short intro that frames the model as writing an executive briefing for a CFO or VP of Finance.

User message

Composes the buyer situation, per-vendor breakdown, and requirement list into a 3-5 sentence summary ask.

Step 6

Follow-up questions

4-5 targeted questions the buyer should ask in vendor demos, prioritizing actual gaps and points of divergence.

Role

Bridges the report into the buyer's next action. Produces consultant-grade questions, not brochure-answerable generic ones.

Model

Claude Sonnet (claude-sonnet-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • All requirements
  • All findings
  • Vendor list
  • Process description

Outputs

  • Array of 4-5 follow-up question strings

System preamble

Short intro that frames the model as writing follow-up questions for a finance leader post-report.

User message

Asks Claude for 4-5 consultant-grade questions targeting the report's actual gaps and divergences.

Step 7

Answer a follow-up question

When a buyer asks a specific question after the report exists, answers it grounded in the report's findings and their original scenario.

Role

The report's interactive layer. Must stay grounded in the evidence already gathered, be honest about uncertainty, and name the operational impact of gaps.

Model

Claude Sonnet (claude-sonnet-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • A single question from the buyer
  • The full report (requirements, findings, process description, vendors)

Outputs

  • An answer (2-4 paragraphs) plus a list of sources drawn from the report's findings

User message

Grounds a single follow-up question in the existing report's findings and writes a consultant-tone answer.

Step 8

Validate and generate a new vendor

When a new vendor is added, validates it is a real product in the given category and generates 15 seed capabilities for it.

Role

The admission gate for new vendors. Prevents bogus additions and produces a consistent baseline capability profile so the new vendor participates in reports immediately.

Model

Claude Sonnet (claude-sonnet-4-6)

Tools

None

Sources consulted

None (pure reasoning)

Shared blocks composed

None

Inputs

  • Proposed vendor name
  • Target software category

Outputs

  • Either a rejection with reason, or a GeneratedVendorData with 15 capabilities

User message

Validates a proposed vendor belongs in the category and generates 15 seed capabilities.

Think something is wrong? Suggest an edit.

We treat this pipeline the way good publishers treat editorial standards: written down, open, and revised when someone makes a better argument. Every accepted and rejected suggestion is logged with the reason, so you can see what changed and why.

Last updated: 2026-05-03. A structured in-app dispute flow is in progress; until it ships, email is the channel.