A case study from Artovnia's production, how a default Postgres setting that nobody knew was enabled, added 2 seconds to every cart mutation.
TL;DR
Refactoring the checkout flow at Artovnia, removing manual confirmation of each step in favor of automated, reactive behavior, revealed it was slow on production (add to cart: 2.4–3.7 s, shipping selection: 2.5–2.8 s), while locally the same operations, the same data, ran in under half a second. The culprit wasn't the code, a missing index, or data volume. It was Postgres JIT, a query compilation mechanism that, when it misestimates row counts, can cost more than it gains. In my case: compilation took ~2 seconds, and the query itself executed in 1.4 milliseconds.
On top of that, nobody consciously enabled this JIT. It's a default Postgres setting since version 12, and there's nowhere to see it until you ask the database directly. Also locally I work on Windows with virtual linux, while production is on Linux with official llvmjit image.
Symptom
Production metrics spoke for themselves:
- POST /store/carts/.../line-items (add to cart) — 2.4–3.7 s
- POST /store/carts/.../shipping-methods (shipping selection) — 2.5–2.8 s
- POST /store/carts (cart creation, no promotions) — 255 ms, including database: 11 ms
That last line is key. Cart creation, an operation without promotion logic, was fast. Everything that touched promotion recalculation was slow. That narrowed down the problem before I even ran EXPLAIN.
Locally, the same requests with the same (or very similar) data ran in under half a second. Same code. Same number of promotions, rules, values. So it wasn't the algorithm or N+1 in the classical sense, it was something environmental.
Before I Found the Real Lead
Before I even reached for EXPLAIN, I did what seemed most obvious: I tried to improve the perceived smoothness of checkout from the frontend side. The "Add to cart" button stopped waiting for the full backend response, and showed confirmation immediately, using state that was already going into the global cart store. The transition from the address section to the shipping section started happening optimistically instead of waiting for a hard confirmation of the save. The shipping section got a skeleton instead of a blank screen while waiting.
These changes were worth making in their own right, checkout stopped feeling frozen and the user saw a reaction to their action immediately. But these were perception fixes, not actual time fixes. The step responsible for promotion recalculation still took the same ~2.5 seconds on the server side, no matter how cleverly the interface masked it. Optimistic UI shortens the time the user waits idly, it doesn't shorten the time the server actually needs. Those are two separate measurements, and it's easy to confuse them if you're only looking at "how fast it feels now".
Only when I stopped trying to hide the slowness with the interface and started measuring it directly on the backend, the problem stopped being a mystery.
How I Tracked It Down
I have a per-step instrumentation panel (timeStep) in the project that breaks down each request's time into processing stages. For the most affected endpoint, it looked like this:
POST /store/carts/.../shipping-methods total 2.70s
promotion_refresh 2.57s (95%)
final_graph 50ms (2%)
add_method 36ms (1%)
resolve_cart 29ms (1%)
loyalty 8ms (0%)And two more requests, same pattern:
total 2.79s → promotion_refresh 2.65s (95%)
total 2.53s → promotion_refresh 2.38s (94%)One step, promotion_refresh, consistently consumed 94–95% of the entire request time. The rest of the process, resolving the cart, adding the shipping method, loyalty, building the final graph, was a few dozen milliseconds in total. Zero suspicion in that direction.
One level down: SQL query telemetry for the same request showed db.query.count=75, db.query.total_ms≈2409, db.query.max_ms≈2068. So out of 75 queries executed during promotion recalculation, a single query was responsible for almost all of the time.
The slow-query log identified it precisely, a query loading promotions along with the entire relationship tree: the application method, target rules, "buy X" rules, rule values, campaign, campaign budget. A typical wide SELECT with a cascade of LEFT JOINs, the kind MikroORM generates under Medusa's hood when loading an object graph.
That's when I reached for EXPLAIN (ANALYZE, BUFFERS) directly on production, and that's where the real story begins.
The Culprit: Postgres JIT
The query plan showed a section at the end that you don't normally see in typical debugging:
JIT:
Functions: 107
Timing: Generation 8.7 ms, Inlining 23.1 ms,
Optimization 1212.2 ms, Emission 810.3 ms,
Total 2054.3 ms
Execution Time: 2056.2 msOut of 2056 ms total execution time, 2054 ms was JIT compilation alone. The actual table scans and joins, visible in the individual plan nodes, took under 2 ms. In other words: Postgres spent 99.9% of the time compiling code, to execute a query that was lightning-fast anyway.
For comparison, the same query with JIT disabled:
Settings: jit = 'off'
Execution Time: 1.453 msSame query plan. Same nodes, same costs, same buffers. The only difference: 2056 ms versus 1.4 ms.
Why JIT Fired at All
Postgres doesn't compile every query, that would be pointless for simple operations. It fires JIT only when the estimated plan cost exceeds a threshold (jit_above_cost, default 100,000). If the cost is even higher, it adds more expensive variants: LLVM optimization and function inlining (thresholds jit_optimize_above_cost and jit_inline_above_cost, both default 500,000).
The problem with this query: the planner estimated the intermediate join result at 1,151,348 rows. In reality, 103 came back. An error of eleven thousand times. The estimated cost of the entire plan came out at over 2.4 million, well above all three thresholds at once, so Postgres fired JIT at full power: compilation, optimization, and inlining of 107 functions, for a query that in reality returns a hundred-odd rows in under a millisecond.
Where does such an absurdly inflated estimate come from? The query has a cascade of about ten LEFT JOINs, including several one-to-many relationships passing through bridge tables, plus a WHERE condition with an OR across two different columns (code IN (...) OR id IN (...)). Postgres's cost model assumes statistical independence between successive joins in a chain, it multiplies selectivities together instead of calculating their actual correlation. The longer the join chain, the more this error compounds exponentially. This isn't a matter of stale statistics, it's a fundamental planner limitation with wide, multi-table queries of the kind ORMs regularly generate when loading complex object graphs.
For the Uninitiated: What JIT Actually Is
If you don't work with databases day to day, it's worth clarifying what this is all about, because at first glance it sounds like something that should only speed things up.
Normally Postgres executes a query by interpreting its plan step by step, a bit like someone reading a recipe aloud and carrying out each instruction separately, from scratch each time. JIT (Just-In-Time compilation) is an attempt to speed this up: instead of interpreting repetitive fragments (e.g. filter conditions, reading values from a row), Postgres compiles them into native machine code using LLVM, the same compiler framework as Clang or Rust.
This makes sense for heavy, analytical queries: aggregations over millions of rows, complex calculations repeated hundreds of thousands of times. There, the compilation cost (on the order of tens to hundreds of milliseconds) is spread over a huge number of executions and genuinely pays off.
The problem: the compilation cost must be paid upfront, regardless of how much the query actually does. If Postgres incorrectly estimates that a query will be "heavy", as in my case, where it expected a million rows instead of a hundred, it compiles code as if this were a large analytical operation. And then the query executes in microseconds anyway, because it was never actually heavy. I paid to build a highway to drive it once, twenty meters.
Why You Can't Just See It in Settings
Here a question naturally arises: if it's so problematic, why did someone enable it?
The answer is less satisfying than "someone made a configuration mistake": nobody enabled it. Postgres doesn't have anything like a settings tab with toggles. All configuration lives in a flat text file (postgresql.conf) with several hundred parameters, no interface, no descriptions in a UI, nothing that would notify an administrator "hey, this feature is active and may affect your queries".
On top of that, there's an additional trap when reading this file. A commented-out line in postgresql.conf:
#jit = on # allow JIT compilationdoesn't mean "disabled". It means: nobody has overridden this, so the built-in default value applies, and that value is on. This actually misled even the AI diagnostic tool I used along the way to check the hosting config, it read #jit = on as evidence that JIT is disabled by default, because the line is commented out. If even an automated agent fell into this interpretation trap, it's all the less obvious to a human skimming the file.
Hosting platforms like Railway deliberately don't wrap Postgres in a UI settings layer. You get a "vanilla" image, zero configuration on top, so you inherit every default parameter of the Postgres project unchanged, including those that changed between versions. And JIT is exactly that kind of case: in Postgres 11 it was off by default (the feature was new), from Postgres 12 it's on by default. I was on Postgres 16. I got the factory setting from four consecutive major versions, with no human decision along the way.
The only way to find out is to ask the database directly:
SELECT name, setting, source FROM pg_settings WHERE name = 'jit';The column source = 'default' says it clearly: this is the built-in value, nobody has overridden it anywhere.
Does anyone normally look in there? Most web developers have never had a reason to think of a database as anything other than a black box sitting behind a connection string.
In this case, the exception was a particularly complex LEFT JOIN generated by Medusa.
Fix: How to Disable JIT
Disabling JIT at the database level:
ALTER DATABASE "nazwa_bazy" SET jit = off;This is not a configuration file change, it goes into Postgres's system catalog and doesn't require a server restart. However, it only takes effect for new connections. Here is the first deployment trap: if the application holds a pool of already-open connections (and every ORM does), those connections carry the old setting for the rest of their lifetime in the pool. Changing the database without restarting the backend changes nothing in practice, you need to redeploy or restart the application to force new connections.
The second trap: ALTER DATABASE ... SET is a manual change on a specific database instance, not recorded anywhere in code or migrations. If the database is ever recreated from scratch, new environment, disaster recovery, migration to a different host, that setting disappears without a trace. Safer in the long run is a version that travels with the application configuration, as a startup parameter in the connection string:
postgresql://user:pass@host:port/db?options=-c%20jit%3Doff(that is, URL-encoded -c jit=off, passed to libpq on every connection, regardless of whether the database remembers anything).
There's also an intermediate option for someone who doesn't want to give up JIT entirely in case of genuinely heavy analytical queries in the future: instead of disabling it, drastically raise the activation threshold (jit_above_cost), so that incorrectly estimated transactional queries never exceed it, while truly heavy queries do.
The Result
I don't have hard before/after numbers from production traffic. After deploying the fix, the times for those requests dropped below the threshold at which my panel even registers a request as "slow". That in itself is fairly telling: the endpoint that previously systematically appeared on the list of slowest requests with times of 2.5–3.7 seconds, simply stopped showing up there after the fix.
What I do have measured concretely is the EXPLAIN on the identical query: 2056 ms with JIT, 1.4 ms without. The rest of the difference between local and production environments is no longer JIT, just the real network round-trip between the application and the database across 75 queries that make up the promotion recalculation, a separate topic for optimization.
The General Lesson
If you use an ORM that generates wide queries with many JOINs when loading an object graph (MikroORM, TypeORM, Prisma with nested include), you have a young database with a small number of rows in key tables, and you're on Postgres 12 or newer, it's worth checking this before spending time hunting for the fault in indexes or N+1:
EXPLAIN (ANALYZE, BUFFERS) <your query>;If the output contains a JIT: section with a time comparable to or greater than the actual query execution time visible in the plan nodes, it's not your query that's slow. It's compilation that should never have fired in the first place.
This problem was purely infrastructural and configurational, it didn't require changing a single line of application logic. Sometimes the most expensive bug in checkout isn't in the checkout code.