Cookie Monster Database

My office building is located across the street from a shopping center, and for some reason, the one store that always catches my eye is Crumbl Cookie. When I was first introduced to Crumbl Cookie, I enjoyed their cookies but was more fascinated by their business model. Over the years, I’ve become disillusioned with their cookies and think they can sometimes taste like sugar-coated cardboard. There is one cookie, though, that still captures my attention and imagination, the Almost Everything Bagel Cookie. Most people are appalled at the thought of an Everything Bagel flavored cookie and are even more wary of trying said cookie. Unfortunately, I have not seen this cookie, with its wonderful blending of savory flavor and the sweetness you expect from a cookie, back at any local Crumbl in quite some time. Periodically, while sitting through a meeting, I think about the Almost Everything Bagel Cookie and sometimes even head to the Crumbl website to see if it is one of this week’s flavors. For the last 3+ years, the answer has always been no. With so many stores I always wondered if I could create a tool and find if any store carried the Almost Everything Bagel cookie.

During my most recent investigation of the Crumbl website, I did some digging through the structure of the site, as I tend to do. One of the first things that popped into my mind was how organized the website was, with the “Select a Location” page — https://crumblcookies.com/order/carry_out — having all the stores conveniently listed alphabetically. The wheels began to turn, and I wondered how easily all this data could be extracted. When digging a little deeper into the store-level view, I discovered that it too was neatly structured, and each store’s weekly menu was found using the following URL format: https://crumblcookies.com/order/carry_out/41ststreet, with the slug ID being at the end of each URL. I realized this data could be scraped, and I could find out whether the Almost Everything Bagel Cookie had completely disappeared or had simply not returned to any of the Crumbl stores near me.

With a basic understanding of how the unique store URLs are structured, I had to decide how to handle creating my Cookie Monster database. I knew web scraping would probably be the route taken, but I did some further testing to see if the website had an API I could interact with (this happens more often than you might think). Unfortunately, it did not, so I started writing the code to web scrape the site. I then had the idea of letting an LLM do this task for me. I use LLMs, mostly Anthropic’s Claude, on a daily basis, usually to help clean up and improve my code. Recently, I have seen LLMs capabilities continue to increase to a point where they can write an entire codebase on their own. I thought this Cookie Monster database would be a perfect way to test the model’s capabilities, as well as familiarize myself with the technology.

I used Claude Code to complete this task. I started by providing an in-depth prompt that included my goal, my initial research of the URL structures, and a minimal viable product. To my surprise, Claude took my task and ran with it. I provided Claude with a directory to work in, and it asked me to install Playwright to handle the web scraping. Throughout the process it was conversational, Claude would generate code, I would run and test said code, and then apply any changes I wanted to make. From start to finish, this process took a little under one hour, which is far faster than I could have ever completed this task on my own. This goes to show the beauty and power of these models: I could have written this code myself and it might have taken me a full day at best, but by leveraging Claude, the task was completed far quicker.

The code generates a database with two tables: one to track all the stores, and a second to track which flavors are available per store, per week. The code can be found here. Although I did make some adjustments to the data structures, I will take no credit for this code, as it was generated by an LLM. The code base is powerful and has many different use cases, from tracking the growth and expansion of Crumbl Cookie franchises (and maybe even closings of stores) to analyzing which special cookies of the week appear most often throughout the year. But to answer the more important question, what about the Almost Everything Bagel Cookie? I found a single store in Florida that had the Almost Everything Bagel Cookie during the second week I ran this process! The only difference is that it now comes in sandwich form. Maybe Crumbl is using it as a test product, but I remain hopeful and hold out for the return of the Almost Everything Bagel Cookie to a store near me.

SELECT *
FROM   menu_items
WHERE  lower(cookie_name) LIKE '%everything%'