A creative team can win the wrong experiment. A video may lead a test group while the proposed production package still lacks evidence that it improves the existing campaign. A package may improve an acquisition metric while the acquired cohort has not matured enough to support a larger commercial commitment.
Google App campaigns experiments should be chosen by the decision they need to support. Directional experiments compare video assets or groups for creative prioritization. App Uplift experiments examine the effect of adding video assets to an existing campaign. Both currently document Android eligibility limits. Neither automatically establishes incremental business revenue, profitability, or the result a different market will achieve. Verify the specific workflow in the account before commissioning the test.
This guide is for app founders, heads of growth and UA leads deciding which creative experiment to fund, what an agency should deliver, and when the answer is too weak to authorize the next step. Platform guidance was checked on November 13, 2026. The decision framework and fictional example below are editorial analysis, not Sharply Labs client results.
Which Google App campaign experiment fits the decision?
Start with the sentence the team wants to write after the test. “Produce more executions of this concept” and “add this video package to our established campaign” require different comparisons.
Pending decision Relevant method What the comparison can support What still needs separate evidence ------------ Which video concept or execution deserves further production? Directional experiment Prioritization among the selected test groups under the declared conditions Transfer to the main campaign, another market, or mature user value Should an existing campaign receive an additional video package? App Uplift experiment The campaign-level effect of adding the tested assets under that setup The isolated contribution of every video and incremental business profit Which assets are receiving delivery in normal operation? Ordinary asset reporting Portfolio diagnosis and candidates for testing A controlled verdict about the effect of replacing an asset Does paid activity create additional business outcomes? An appropriate incrementality study An estimate tied to its actual control, outcome and design Conclusions beyond the tested population, period and measurement coverage
Google describes the first two jobs in its Directional overview and App Uplift best practices. The last two rows are analytical boundaries. A creative-treatment experiment does not contain an advertising-withheld control merely because its interface uses the word “uplift.”
Use the Google App campaign creative-assets guide for portfolio coverage and observational ratings. Use the separate geo holdout guide when the buying decision concerns a business incrementality engagement. Keep those jobs distinct in the agency proposal.
Verify eligibility before writing a production brief
As currently documented, Directional experiments are available for Android, video-only App campaigns for installs. App Uplift Health Check also identifies iOS as unsupported. These are product-specific boundaries, not permission to infer support from the existence of a general Experiments menu. See the Directional eligibility guidance and Uplift Health Check.
Google’s general pages are not completely consistent: the Experiments overview includes App campaigns in a custom-experiment description, while the dedicated custom-experiment setup page excludes App campaigns and directs readers to App asset experiments. That conflict is a reason to verify the exact app-specific workflow and available controls. It is not evidence that an advertiser can test arbitrary App campaign bidding, targeting or landing-page changes through a generic custom experiment.
Ask the operator to record the app, operating system, campaign subtype, available experiment type, selected base or test campaign, and any eligibility messages. Save a dated settings record. If the required path is absent, clarify eligibility with Google or revise the study; copying a Search experiment tutorial does not resolve the mismatch.
Account access and a sound question also do not establish feasibility. A campaign with unresolved policy, event-quality or delivery problems may need repair first. The campaign-goal readiness guide helps separate a creative question from an optimization-signal problem.
Directional experiments: define exactly what competes
Google documents a setup with a new video-only install campaign, one active ad group and at least two videos, with no other active test in the campaign. Its overview recommends a geography excluded from the main campaigns to reduce overlap. Record the proposed market and its exclusions before approving spend (Directional setup, overview).
A separate test market improves operational separation but introduces a transfer question: how similar is this audience, language and product context to the market where the team wants to deploy the result? A concept that explains an unfamiliar category may behave differently where the product is already well understood. Geographic separation is a design condition, not evidence of universal creative preference.
Preserve a single interpretable difference
Write a hypothesis at the level the team can act on. For example: a product demonstration versus a narrated explanation, with the same offer, duration and orientation. If both the proposition and the format change, the package comparison may still be useful, but the result cannot identify which change mattered.
Google explains that test groups are randomly selected to enter auctions, with similar serving preference and pacing. It also warns that orientation, length and quality can produce different impression volumes because placement eligibility differs. Equal opportunity to enter an auction should not be reported as guaranteed equal exposure (how Directional experiments work).
That matters when a report names a leader. Record exposure, cost and the selected success metric together. A format decision can legitimately include the inventory available to that format. A claim about a hook alone needs tighter matching.
Inventory shared assets before launch
Google’s setup guide says video assets not assigned to a specific test group are used across all test groups. The asset register must therefore identify both the assigned differences and the shared material (Directional setup).
Suppose a team labels two groups “product proof” and “creator story,” but leaves another product demonstration unassigned. That shared video can appear across groups. The tested comparison includes that shared background; it is not a clean comparison of two entirely separate creative libraries. Record exact asset IDs and group membership instead of relying on a memorable test name.
The same guide says early results may be visible in two to three days and the default end date is fourteen days after the start. It also explicitly gives no strict duration or budget recommendation. Treat visibility and defaults as interface behavior, not a promise of a decision-ready sample. The leading group can change as the test continues.
App Uplift: evaluate the added package
App Uplift addresses whether adding video assets changes the existing campaign’s performance. When several videos are added, the treatment is the package under the experiment’s delivery conditions. Google allows individual-video reporting, while describing the selection of a video winner from that reporting as directional (App Uplift interpretation).
The commissioning question should name the base portfolio and the addition. “Does this package improve our declared acquisition decision?” is more useful than “Are these videos good?” Preserve the original campaign’s asset inventory, event definitions, destination, market and intended constraints in the brief.
Google’s creation instructions document split, confidence, goals, dates and synchronization controls. Synchronization needs an owner: the operator should know which changes propagate and verify consistency rather than assume the base and trial remain matched after every edit.
The current instructions recommend a balanced split and an 80% confidence setting; those are Google product recommendations. They do not replace a business decision about acceptable error, downside exposure or the cost of a false positive. A higher-confidence setting can need more evidence, and a more permissive setting does not repair unreliable event data.
Resolve health warnings without automatic budget increases
Health Check covers operating system, budget limits, bids, available budget, confidence and duration. An issue can explain why a test may struggle to become conclusive (Health Check documentation).
Treat each warning as a feasibility dependency. Reconcile the proposed remedy with the actual bidding strategy and approved spending boundary. “Increase budget” is not an editorial or agency default: the owner may prefer to narrow the question, defer the experiment or improve the assets first. A test that requires unacceptable exposure is unsuitable even if the interface offers a way to start it.
Use an acceptance register before money moves
A useful experiment brief records what makes the work acceptable before anyone sees a leading result. The following register is a proposed operating tool, not a platform requirement or a claim about a proprietary Sharply Labs method.
Boundary Evidence to retain Decision that remains blocked when it fails --------- Eligibility and operation App/OS, experiment path, campaign IDs, warnings, approved exposure, exclusions and owner Launching the intended platform experiment Treatment integrity Base inventory, assigned/shared assets, one declared difference or explicit package, destination and change log Attributing the comparison to the intended creative difference Measurement and maturity Success metric, event mapping, reporting range, attribution treatment, data gaps and cohort age Calling the observed metric the required business outcome Action and transfer Adoption rule, downside guardrails, deployment market, rollback owner and outstanding questions Expanding production or campaign scope beyond what was tested
Name the decision owner for each row. Creative production cannot certify event mapping; analytics cannot silently authorize media exposure; an agency cannot infer that a product release stayed constant without a responsible product contact.
For downstream checks, compare cohorts at equivalent maturity and preserve the limitations of the attribution data. The app cohort-analysis guide explains why a newer cohort’s incomplete value should not be compared directly with an older cohort’s longer observation window.
A fictional example: the cheaper install does not settle production
Consider an Android subscription app deciding whether to commission six further executions of a concept. These are invented numbers for arithmetic, not benchmarks, a statistical test result or client evidence.
In an illustrative comparison, group A spends $2,400 and records 800 installs. Group B spends $2,400 and records 960 installs. Their observed CPIs are $3.00 and $2.50 respectively. On that measure alone, B looks more attractive.
Now suppose an independently reconciled, equally mature activation view records 160 qualified activations for A and 144 for B. The observed cost per qualified activation is $15.00 for A and about $16.67 for B. The direction reverses because the business denominator changed. These figures still lack uncertainty estimates and do not establish a causal difference in user quality.
The next decision depends on the predeclared purpose. If the test was explicitly screening install-oriented concepts, B may earn another bounded test. If the decision required qualified activation, the team cannot announce B as the commercial winner from CPI. If activation cannot be linked reliably to the comparison groups, mark that outcome unavailable; do not distribute campaign totals between groups by impression share.
Suppose the six additional executions would cost a fictional $6,000. Add that production commitment to the decision memo separately from media cost. The experiment does not need to “pay back” the entire production invoice during its observation window, but management should understand what uncertain learning is being used to authorize that invoice.
A sensible memo might approve one revised execution for follow-up, defer the larger package and ask analytics to resolve the activation comparison. That is a bounded production decision. It makes no claim that the platform has identified a permanently superior concept.
Interpret the reporting window and an inconclusive result
App Uplift reporting compares full days within the experiment dates and selected reporting range. Google recommends excluding the first five to ten days when interpreting results to reduce the effect of campaign learning (monitoring guidance). Keep the chosen range in the handoff; two screenshots with different windows may describe different comparisons.
Google’s Uplift best practices recommend a thirty-day run where possible, discuss campaign interference, and advise on settings and asset changes. These are product-specific planning recommendations, not a universal sample-size calculation or a guarantee of significance. Conversion delay, event frequency, the tested effect and business risk still determine whether the evidence supports the intended action.
When a result is inconclusive, classify the reason before spending again:
Operation failed: an eligibility, approval or delivery issue prevented the intended comparison. Repair that dependency before commissioning another creative variant.
Exposure was insufficient: the assets did not receive enough usable traffic. Reassess feasibility and approved exposure; additional time is an option only when it can answer the question.
The treatment was ambiguous: shared assets, changed offers or simultaneous edits prevent the planned interpretation. Revise the comparison and preserve the failed test as a learning record.
Business evidence is immature: acquisition reporting exists, but activation or value is incomplete. Wait for the declared maturity or limit the conclusion to the observed outcome.
No decision-relevant difference is established: the test ran, but the evidence does not justify the proposed commitment. Retaining the current portfolio can be the appropriate action.
Avoid presenting an inconclusive result as proof of equivalence. Also avoid selecting a different success metric after seeing which one favors the preferred creative. Any exploratory finding should be labeled and assigned a new validation question.
What should an app-growth agency deliver?
A proposal should identify the pending business decision, supported experiment type, feasibility dependencies and exact treatment. It should separate media budget, creative production, measurement work and management fees. Ask for a launch register, a change log, a reporting-window record and an adoption memo with unresolved limitations.
The adoption memo should state what happened, which inference is supported and which action follows. A Directional leader might justify further creative development. An Uplift result might support applying a tested package in comparable conditions. Neither requires the memo to promise lower future CPI, a profitable cohort or successful transfer to every campaign.
Sharply Labs’ mobile app growth service covers acquisition, creative and measurement work with responsibilities and deliverables scoped around available evidence. The broader growth services page explains the connected paid-acquisition scope. Platform availability, engineering work and deeper analysis still need explicit scoping.
Bring one pending creative decision
For an established app team evaluating a Google App campaign creative test, bring a redacted campaign and asset inventory, experiment availability, event definitions, market overlap, approved exposure and the production decision waiting on evidence.
In an initial Sharply Labs consultation, the conversation can identify which experiment fits, what evidence is missing and who should own the next step. You receive a prioritized decision agenda; detailed design, account analysis, implementation and operation are separately scoped. The conversation does not promise eligibility, conclusive results, incremental revenue, lower acquisition costs or a completed account audit.