AI just made experimentation almost free. Ruby on Rails makes shipping experiments trivial. Very few people are putting these two things together.
Most teams use AI to write code or draft copy. Fewer use it for the thing that really pays off, generating credible alternatives.
Experiments used to be expensive
A/B testing was always a good idea. It was also always a hassle.
You needed copy variations someone had to write. Design variations someone had to mock. An engineer had to wire a feature flag or plug in a third-party tool. A PM had to define a metric. Two weeks later you got a winner and shipped it. Maybe.
That cost meant most teams only experimented on the big things: pricing, the hero section, checkout. The hundred smaller copy and layout decisions, the ones that actually compound, were made by whoever felt most confident in the meeting.
AI ate the expensive part
The bottleneck was always generating plausible variations. That's gone.
"Give me five versions of this hero section. First one focuses on time saved. Second on cost. Third on trust. Fourth is aggressive. Fifth is minimalist." One prompt. Five variations, each with its own copy and its own visual direction.
Same for onboarding flows, empty states, confirmation dialogs, pricing tables, email subject lines, error messages. The creative work that used to eat a sprint now happens while you're making coffee.
If you've ever looked at your homepage and thought "I bet a different angle would convert better but I'll never get time to find out"... you now have time to find out.
Rails was built for this
Here's where the stack matters. Rails has great support for ERB variants, and it pairs well with any feature flag library (Flipper is the usual pick). You can do this:
# app/controllers/pages_controller.rb
def home
request.variant = :b if Flipper.enabled?(:new_hero, current_user)
end
# app/views/pages/
# home.html.erb <- control
# home.html+b.erb <- variant B
That's the whole thing. Rails picks the right template based on the variant. The feature flag decides who sees what. You can gate by plan, by country, by cohort, by random bucket — any business logic you can express in Ruby.
Rolling back is as cheap as flipping the flag off. No deploy needed. That's what makes experimenting safe: if a variant tanks, it's gone in seconds.
Measuring what your users actually prefer
Ahoy (or any event tracker you like) makes this part easy. Emit an event when a user sees a variant, another when they complete the action you care about, and you have conversion per variant.
ahoy.track "viewed_hero", variant: request.variant
ahoy.track "clicked_cta", variant: request.variant
Query the results in a rake task, a dashboard, or just the console. Pick the winner. Kill the flag. Ask AI for the next five variations. Repeat.
One warning if you are new to this: do not call a winner the moment one variant pulls ahead. Evan Miller's How Not To Run An A/B Test is the classic two-minute read on why that instinct will fool you. Worth the detour before you start shipping experiments every week.
The loop is the point. Each turn of the wheel costs almost nothing, and each turn teaches you something real about your users.
WordPress was great. Now it's a drag.
WordPress was the right answer for a long time. The plugin ecosystem made it easy to assemble a site without writing much code.
That same ecosystem is now the ceiling. If I want to test a new pricing table, a redesigned checkout, or a different onboarding email, I don't want to hunt for a plugin, pay for a SaaS add-on, or wedge JavaScript into a page builder. I want to ask AI for five variations and ship them behind a flag this afternoon.
On a Rails app I can. On WordPress, I'm fighting the platform before I even start.
Pick tools that don't tax every experiment you want to run.
Summary
The moat used to be the infrastructure for experimentation: writers, designers, engineers, tracking, flags. AI removed the cost of generating variations. Rails removes the cost of shipping and rolling them back. If your platform makes experiments hard, every small decision on your site is a guess you are not testing. That is the growth you are leaving on the table.
