Semantr reads the Bedrock invocation logs you already have, groups the invocations back into conversations, and works out what kinds of work your organisation is actually doing with AI. Not which team spent what — what the money bought.
Set up Semantr so I can see what our organisation is actually using AI for.
Semantr reads AWS Bedrock model invocation logs, groups the invocations back into
conversations, works out what kinds of work we are doing, and writes a report.
Please:
1. Install it: pip install semantr
2. Find our Bedrock model invocation logs. They are usually delivered to S3 — look for a
bucket with an AWSLogs/<account>/BedrockModelInvocationLogs/ prefix. Check the regions
we actually use. If model invocation logging is not switched on anywhere, tell me that
rather than guessing, because there will be nothing to read.
3. Price the run before spending anything:
semantr analyse <the s3:// path> --classifier-model bedrock/anthropic.claude-3-5-haiku-20241022-v1:0 --dry-run
Tell me the estimate. It should be well under a dollar. If it is not, stop and say so.
4. Then run it for real:
semantr analyse <the s3:// path> \
--classifier-model bedrock/anthropic.claude-3-5-haiku-20241022-v1:0 \
--embedding-model bedrock/amazon.titan-embed-text-v2:0 \
--json-out semantr-report.json
5. Open semantr-report.html and tell me:
- the three biggest use cases by spend, and what they actually are
- anything in there we would not have predicted
- the cost improvements it suggests, and whether they look right to you
Notes:
- It needs bedrock:InvokeModel on those two models, plus read access to the log bucket.
- Prompt text never leaves the machine. Nothing is uploaded unless I ask for it.
- If we do not use Bedrock, ask me which provider we do use before changing anything.
No signup, no account, nothing uploaded. It runs in your own environment and writes a file. Sign up later if you want history and a shareable link.
Cost dashboards report spend by team, key and model. That answers an accounting question. It does not answer the one you are being asked.
"$40k on Bedrock last quarter" is a number, not an answer. Spend by API key does not tell anyone whether the money bought customer support or someone's side project.
Every team knows what they built. No one knows what the other eleven teams built, and the list nobody maintains is always out of date.
Classifying every invocation with an LLM costs tens of thousands of dollars at real volume. Which is why nobody does it, and why nobody knows.
Everything before the naming step is free. The naming step runs about 150 model calls whether your corpus is a million invocations or fifty million — which is what makes this cost cents instead of tens of thousands.
Bedrock logs are stateless — a twenty-turn conversation is twenty unrelated records. But they are self-describing: each invocation contains the one before it. That makes rebuilding conversations an exact join, not a guess.
2,761 invocations → 1,007 conversations
Similar conversations are grouped, and each group is named from its own contents. There is no fixed taxonomy and therefore no other bucket — the categories come from what your organisation is actually doing, including the things nobody thought to ask about.
1,007 conversations → 221 distinct prompts → 21 named use cases
Spend, tokens and time roll up against each discovered use case, with the cost improvements that follow from them. Cost per conversation, not per request — a twenty-turn session is one unit of work.
$1.10 across 21 use cases · analysis cost $0.03
Self-contained: inline styles, embedded data, zero external requests. It opens in any browser, prints cleanly, and can be emailed as-is. A CISO can read the whole thing before deciding whether to trust it.
| Use case | Tags | Conv | Spend |
|---|---|---|---|
| Customer Support Responses | non-coding repeatable | 206 | $0.62 |
| Invoice Data Extraction | non-coding repeatable | 238 | $0.03 |
| Pull Request Review | coding repeatable | 131 | $0.01 |
| Churn Analysis Queries | coding experimental | 44 | $0.07 |
| Database Script Development | coding experimental | 34 | $0.01 |
Every use case above was discovered, not configured. Two of them were things the team running it did not know existed.
Coding vs not, and repeatable vs experimental — so you can see at a glance how much of your spend is automation and how much is people trying things. Cadence is measured from the data, not guessed by a model.
Model right-sizing, prompt caching, retry loops, untagged spend — each with the annual saving worked out, so it is a decision rather than a suggestion.
Bedrock invocation logs contain full prompts and completions — the most sensitive data class a company produces. Semantr is built so that the awkward conversation never has to happen.
The CLI reads your logs in your account. With no API key configured it makes no network calls at all — not a version check, not a ping. Embeddings can run locally too, so only ~150 naming calls ever touch a provider.
Sign up and the CLI can push counts, cost, and the names it discovered. Prompt text is stripped before anything is sent, and the egress schema is published in the README rather than buried in a policy.
You get the analysis and the report without an account, forever. The paid tier is history and trends — which a file genuinely cannot give you.
Running Semantr costs you a few cents of inference per run, paid to your own provider. We do not mark that up and we never see it.