Wayne Workman

Note: This is a personal project done on my own time with my own resources. No connection to my employer.

So I've been experimenting with AWS Bedrock lately and this morning I realized I needed a consistent way to extract code from the Converse API responses, especially with cross-region inference profiles. Thats when I discovered this really annoying limitation. Turns out cross-region inference doesnt support native tool usage through the Converse API. Like, at all. Which is frustrating because the whole point of cross-region is better availabilty and not being stuck when us-east-1 has issues (which happens more than you'd think).

Anyway I spent the day building a workaround that actually works pretty well and figured I should share it. Just put the whole thing up on GitHub a few hours ago if you want to check it out: bedrock-crossregion-tool-workaround. Figured I'd write this up while its still fresh in my mind, because honestly this could save people a lot of time and money. Especially if your running multiple projects or, you know, have a team thats experimenting with different models all the time.

The Real Value of Cross-Region Inference

Ok so here's the thing about AWS Bedrock's Converse API that really caught my attention when I was reading up on it. I read that before the Converse API, switching between different AI models was apparently a nightmare because each model had its own completely different input and output formats. People were writing hundreds of lines of wrapper functions just to normalize everything. When I saw that the Converse API provides a uniform interface across all these models, I was like, ok thats what I'm using then. No brainer.

Now with the Converse API? You literally just change one string. The MODEL_ID. Thats it. Want to switch from Claude Opus to Nova Premier to see if the cheaper model can handle your use case? Change one line. This is huge when you think about it at scale... imagine having 50 different automation scripts or tools across your organization. Being able to test them all with a cheaper model by just changing a config value? That could be the difference between a $10k monthly bill and a $60k one.

Cross-region inference profiles make it even better, because now your not locked to a single region. The profile IDs look like us.amazon.nova-premier-v1:0 or us.anthropic.claude-opus-4-1-20250805-v1:0. Notice the "us" prefix? That means it'll route to whatever US region has capacity. No more getting stuck because us-east-1 is having a bad day.

But heres the catch, these cross-region profiles don't support the native tool usage features. Which is kind of a big deal if your trying to build anything that generates and executes code dynamically.

The Workaround That Actually Works

So after digging around a bit, I figured out you can basically teach the model to "use tools" through prompt engineering. Its not as clean as native tool support but it works surprisingly well. And more importantly, it works consistently across different models, which means you can still swap models easily for cost optimization. Took me a few tries to get the prompt format right but once I did, it clicked.

Here's the core concept from my GitHub repo:

# From bedrock_custom_tool_demo.py
prompt = f"""You are an AI assistant with access to execute Python code through a special tool.

TOOL USAGE INSTRUCTIONS:
When you need to execute Python code, you MUST follow this exact format:

1. First line must contain ONLY: TOOL: python_executor
2. Immediately follow with a markdown code fence containing your Python code
3. Do not include any text between the tool declaration and the code fence

Example of correct format:
TOOL: python_executor
```python
import boto3
ec2 = boto3.client('ec2')
response = ec2.describe_instances()
print(response)
```"""

What your doing here is training the model in-context to respond in a very specific format when it needs to generate code. I tested this today with Nova Premier, Claude Sonnet 4, and Claude Opus 4.1, they all follow the pattern reliably if you keep the temperature low (like 0.3). Higher temperatures and sometimes the model gets creative with your format which breaks extraction.

Extracting the Generated Code

Once the model responds in our format, extracting the code is pretty straightforward. Just some basic string manipulation and regex:

# Extract response text from Bedrock
response_text = response['output']['message']['content'][0]['text']

# Check if model wants to use our "tool"  
lines = response_text.strip().split('\n')
if lines[0].strip().upper().startswith('TOOL: PYTHON_EXECUTOR'):
    # Find and extract the code using regex
    code_match = re.search(r'```python\n(.*?)\n```', response_text, re.DOTALL)
    if code_match:
        code = code_match.group(1).strip()
        print(code)

The re.DOTALL flag is important here, it makes the dot match newlines so you can capture multi-line code blocks. Without it you'll only get the first line which isnt very useful.

Why This Approach Scales

Now here's where it gets interesting from a cost perspective. And look, I know talking about cloud costs isn't the most exciting thing, but when your paying for this stuff yourself (or managing budgets), it matters a lot.

# Configuration at top of the script
AWS_REGION = 'us-east-2'
MODEL_ID = 'us.amazon.nova-premier-v1:0'  # Start with cheapest
# MODEL_ID = 'us.anthropic.claude-sonnet-4-20250514-v1:0'  # Mid-tier option
# MODEL_ID = 'us.anthropic.claude-opus-4-1-20250805-v1:0'  # The expensive one

Just uncomment the model you want. Same code everywhere else. I did some rough math on this... for a typical code generation request (around 2000 tokens total), Nova Premier costs about $0.02. Claude Opus 4.1? About $0.12. Thats 6x more expensive.

Now think about that at scale. Let's say you have a team of developers using AI assisted coding tools, or your running automated code generation for testing, or whatever. If your doing 1000 requests a day (which isnt crazy for a decent sized team), thats $20 vs $120. Per day. Over a month? $600 vs $3600. And thats just one use case.

The beautiful thing is, Nova Premier is actually really good at straightforward code generation tasks. You dont need Opus for everything. Its like, why would you use a Ferrari to go get groceries when a Honda Civic works just fine? But without an easy way to swap models, most people just stick with whatever they started with. Usually the expensive option because they want to "play it safe".

Things I Learned The Hard Way

There's definitely some gotchas with this approach that I ran into today while building it:

1. Model consistency varies: Some models are better at following instructions than others. Nova Premier is surprisingly good at it, probably because its trained more recently. Older models sometimes need more explicit instructions.
2. Temperature really matters: Keep it at 0.3 or lower for consistent formatting. I tried 0.7 once and the model started adding helpful explanations between the tool declaration and the code, which broke everything.
3. Context window management: The tool instructions eat into your context window. Not usually a problem but if your generating really long code or passing lots of context, it can add up. Each model has different limits too.
4. Validation is critical: Always check that you actually extracted code before trying to execute it. Sometimes models will explain what they would do instead of actually doing it. Especially if you phrase your request ambiguously.

One really weird edge case, if you ask for something super complex, some models will actually start writing comments about the code before the code fence. Like they cant help but explain themselves first. Thats why the prompt explicitly says "Do not include any text between the tool declaration and the code fence". These little details matter more than youd think.

Looking Forward

AWS will probably add native tool support to cross-region inference profiles eventually. When they do, switching over should be pretty simple since were already using the Converse API. Just add the toolConfig parameter and update the response parsing. The nice thing is all your existing prompts and model selection logic stays exactly the same.

Until then though, this workaround should work great. I tested it today with a bunch of different prompts, from generating terraform configs to writing data processing scripts. Still learning what each model is best at, but the ability to quickly test if a cheaper model can handle the task is going to save me a ton on my personal AWS bill. I can already see the cost difference just from today's experiments. Pretty excited to keep exploring what else I can do with this.

If you want to try this yourself, the code is at github.com/wayneworkman/bedrock-crossregion-tool-workaround. Its MIT licensed so do whatever you want with it. The whole thing is under 80 lines of Python. Sometimes the simple solutions really are the best ones.

Think about it this way, if you can reduce your AI costs by even 50% without sacrificing functionality, thats huge. And with this approach you can actually test and measure which models work for which use cases. No more guessing or just defaulting to the most expensive option because its "safer". You can make data driven decisions about model selection. Scale that across a whole organization and your talking about serious money. Plus the cross-region aspect means better reliability, which is always nice.

Video Demonstration

Want to see this in action? Here's a screen recording where I walk through the entire implementation and show how it works with different models: