Dungeons & Dragons

In my last blog, I talked about my Dungeons and Dragons Notes project. This project is becoming my lab rat, haha. I had an idea to refresh my AWS skills and combine this project with AWS learning.

I will be using only free versions, so some things might be limited, but it is enough for this project for now.

The Idea

The simple version:

• Upload transcript to S3
• lambda triggered -> process the summary
• Save results to S3 and send notification

“But you wanted everything local!!” Yeah, fair point. The whole reason I built it was to keep everything local and private. So actually, let’s build some “fake” data for this purpose.

Critical Role is public and well-known, they upload their videos on YouTube and in fact, they probably have the transcripts available somewhere. Let’s use their sessions to simulate this process.

Step by Step

1. Create S3 bucket with the folders we need (transcripts, summaries). We’re keeping it private, so no public access.
2. Set permissions for IAM user (The most fun part 😑)
3. Test boto3 can talk to our bucket
4. Lambda function - trigger whenever .txt lands in transcripts/
   1. Write the lambda function
   2. The summarization… we will talk about it in a bit
   3. We run a simple test and we can see our /summaries folder now have the test txt file inside
5. Let’s test the automatic trigger - I created a “fake” summary and it will be uploaded to S3. The summary should appear automatically. In CloudWatch, we can see the Lambda logs.
6. Let’s also add SNS to receive an email when our summary is ready
7. This concludes the setup!

Some More Fun? (or not) - back to the summarization

Ollama runs locally, and Lambda can’t access it. We can use API (Claude, OpenAI…), but we wanted to keep it free and local. We can technically deploy it on EC2, BUT - The GPU instances are expensive. The free EC2 is CPU only and it could technically run a small model, but it wouldn’t give us very good results. And nobody wants to see a surprise bill from AWS.

Hence for this use-case, we will keep running Ollama locally, and only use AWS for storage.

Let’s build a little more

When the Python code is executed (manually executed, when the DnD session starts), it will start recording. Currently, we are saving the chunks continuously, and at the end, merging them together into a single file. Then, transcript and summary is triggered automatically. What we can try is to move some of this process to AWS - save the chunks to S3, process them, create a transcript, save to S3 and then get the transcript from S3 back to local computer to perform the summary. Well, it sounds a little tedious, but it is for educational purposes.

So our loop becomes

LOOP until stop:
    read audio chunk
    append to buffer
    IF buffer is full:
        write buffer to local file
        upload buffer to S3
        clear buffer
upload remaining buffer

The recording stops when the user types “stop” command, which triggers the stop_event. There are a few options how to trigger Lambda, but let’s invoke Lambda directly via boto3 after all chunks are uploaded.

We create a Lambda “dnd-merger” function which will merge all mic and sys chunks in order, save the merged audio to S3 and then trigger the transcription. And so again, we have a local whisper model and therefore we would need to deploy it. For this project, we will keep Whisper local, since we have GPU. But yes, it is starting to feel chaotic jumping back and forth, so in real enterprise use-case, this would definitely be different. However even just using S3 for storage and writing some Lambda functions is pretty useful already.

- > lambda merge 
- > save to S3 
- > download from S3
- > local transcription
- > upload transcript to S3
- > local Ollama summary
- > upload summary to S3
- > send email notification 

And with that, the full pipeline was running.

Conclusion

Honestly, the IAM permissions took longer than everything else combined. But that’s AWS. Once everything was wired up, it felt satisfying to watch a file land in S3 and an email arrive a few seconds later without touching anything. It’s a simple pipeline, but it covers the fundamentals, which is enough to get me started and motivated.