Why AI with skills falls short at campaign optimization

In text or code an AI mistake is visible right away and easy to fix. In a campaign a bad decision costs money, can restart the algorithm's learning and may only show up a week later. The stakes are different.

Picture a Monday. Someone exports the last seven days of results, pastes them into a chat with a campaign analysis skill and asks what to improve. The chat sees that a campaign with good ROAS "dropped" over the last two days and advises lowering its budget. But conversions from the last two days have not arrived yet, and the campaign has been in its learning phase for five days. The advice sounds reasonable and is wrong.

Five traps in advertising data

Each is described in the official documentation of ad platforms and model providers. A chat analyzing an export sees none of them unless someone tells it.

Any one of these traps can mislead an analysis. Together, they make a one-off look at an export likely to flag a problem that does not exist or to miss one that does.

How other tools make decisions

Each approach has its place. They share one limitation: the decision depends on one person, one platform or one model call.

How Phanes decides

Phanes does not start by asking a model. It starts with data collected over time and with knowledge written into rules. The language model receives finished numbers and only explains them.

1. 1

   Account history

   Every night Phanes stores each account's daily metrics, so the engine sees trends rather than a single export.

2. 2

   Measurement verdict

   Google Ads, GTM, GA4 and the Meta pixel. Missing data is never treated as a clean result, and suspect measurement blocks conversion-based recommendations.

3. 3

   315 expert rules

   Rules from paid-ads expert practice and Google and Meta documentation, each with a condition, a threshold and a rationale.

4. 4

   Learning-phase gate

   A campaign in learning gets no budget changes or restrictions. Learning window: 10 days on Google, 10 or 14 days on Meta.

5. 5

   A hypothesis with a predicted effect

   Every proposal, and every change people make in the account, records a forecast: which metric should move, in which direction and over what period.

6. 6

   Checked against reality

   After 7, 14 or 28 days, depending on the change type, the engine measures the result with volume and account-trend controls. Confirmations and failures weigh the same.

7. 7

   Knowledge from many accounts

   Conclusions only from at least 3 accounts, none with more than a 40% share. Anonymized, without amounts or content.

Learning from data, not from prompts

Phanes learns the way a good specialist does: it writes down what it expects, then checks what happened. Every rule has a confidence score computed with a Bayesian method from settled trials. A rule that holds up gains priority. A rule that keeps missing loses it, up to and including being switched off. The engine also calibrates separately for each account and change type: where it hits, it acts more boldly; where it misses, it slows down.

Knowledge grows with every account while privacy stays intact. Cross-account conclusions come only from at least 3 accounts, none with more than a 40% share, and only relative indicators travel, never amounts or content. Before a recommendation reaches a card, the engine estimates a probability of improvement calibrated on the history of similar situations across many accounts. Data-driven knowledge never makes automation bolder without evidence from the account whose money is at stake.

Aether: a language model that only knows advertising

This is not an article against AI. Phanes has its own language model, Aether, built for one job: talking about campaigns using exactly what the engine computed. It does not know everything and does not need to. It is narrow, bilingual (Polish and English) and bound by Phanes's rules.

The limits matter most. Aether makes no decisions about the account and never executes changes. It keeps no facts in its weights: it receives advertising knowledge from the Phanes knowledge base at question time, so updating knowledge needs no retraining. It learns form, discipline and style, and it does not learn to guess numbers.

1. 1

   Data without names

   Training uses verdict descriptions, user decisions and expert labels after anonymization: no company names, people, addresses or account identifiers.

2. 2

   Learning the format

   Light fine-tuning teaches the model the answer structure, data schemas and the dashboard language.

3. 3

   Learning from verifiable rewards

   The model generates many answers to the same task, and automatic verifiers reward only those where every number comes from the data, the schema matches and the language is clean.

4. 4

   Release gate

   A new version enters the product only if it beats the previous one on a frozen test set and does not get worse in any task category.

How Aether is built

Aether is not a single large model that knows everything. It is a small set of specialized components, each with one job and its own way of checking quality. The whole cycle, from training to answers in the product, runs on Phanes's own compute infrastructure: client data never leaves our servers and never goes to external model vendors.

The rewards Aether learns from

In reinforcement learning, everything depends on what the model is rewarded for. In Aether, every reward is checked mechanically, never by eye:

• Numbers from the data: every number in the answer appears in the input data, in the same currency. Every foreign number is penalized.
• Schema: the answer passes JSON structure validation.
• Language: the whole answer is in the dashboard language, never mixing Polish and English.
• Honesty: no promises of execution the engine does not provide, no knowledge source names, labels matching the expert, and a citation of a rule that exists.
• Brevity: a penalty for verbosity. Training tasks are picked in the model's difficulty zone (20-80% correct answers from the base), because tasks that are too easy give no learning signal.

What the lab taught us

The biggest lesson is that the verifier matters more than the model. A single bug in the answer extractor understated scores by more than a percentage point and distorted comparisons, so every Phanes verifier has its own tests.

AI with skills is a good assistant. A decision about campaign money, though, should come from a system that remembers the account's history, knows about the learning phase, trusts only verified measurement and gives the same answer on the same data.