High Baseline and how to add proper Control factors

[sonriks6]

Hi there! Got hands on Meridian a few weeks ago and started testing with real data, in automotive industry we run campaigns with a high investment in a short period of time (offline channels mainly) and always on activity in the online channels. Aside that I've found most of the time a pretty high baseline that is not expected (when compared to lightweight_mmm). Played with different control variables as they are confounders of the model but didn't decrease baseline at all. Any ideas or suggestions here?

[jyklly]

Considering different control variables is definitely a good place to start but be sure not to control for any mediator variables, as this could bias your ROI estimates. High baseline equates to low ROI for the media channels. Given the baseline is higher than you expected you may want to include this information in the prior to help regulate the model. If you aren't already you may want to consider informative ROI priors that are higher than what you are currently assuming. Alternatively, you may want to consider setting a total media contribution prior that allows you to set a prior on the proportion of the KPI that is incremental due to all media channels. Please see https://developers.google.com/meridian/docs/advanced-modeling/unknown-revenue-kpi#set-total-media-contribution-prior in the documentation on how to go about this.

[sonriks6]

Yes, I'll check with informative priors, we already have an estimate based on other MMM projects, what I've noticed with Meridian is that the control factors have a lot of influence on the media ROI and baseline. Definitely need to fine tune current approach. Thanks a lot!

[sonriks6]

Hello, tried to incorporate roi_prior for total media contribution following the guidelines but don't know how to inform about it in ModelSpec as this is not well documented in the user guide. Any ideas? A code snippet would be much appreciated.

[jyklly]

No problem. The below snippet builds on what is presented in https://developers.google.com/meridian/docs/advanced-modeling/unknown-revenue-kpi#set-total-media-contribution-prior and is compatible with the Getting Started Colab.

# Calculate total kpi and cost from data
kpi = data.kpi.sum()
cost = data.get_total_spend().sum()

# Create total media contribution ROI prior
p_mean = 0.5  # prior mean proportion of KPI incremental due to all media
p_sd = 0.15  # prior std dev proportion of KPI incremental to all media
roi_mean = p_mean * kpi / cost
roi_sd = p_sd * kpi / np.sqrt(cost**2)
lognormal_sigma = np.sqrt(np.log(roi_sd**2 / roi_mean**2 + 1))
lognormal_mu = np.log(roi_mean * np.exp(-lognormal_sigma**2 / 2))
roi_prior = tfp.distributions.LogNormal(lognormal_mu, lognormal_sigma)

# Set prior in the model spec
prior = prior_distribution.PriorDistribution(
    roi_m=roi_prior
)
model_spec = spec.ModelSpec(prior=prior)

# Create Meridian object
meridian = model.Meridian(input_data=data, model_spec=model_spec)

[christosvis]

I tried the above and when I run the model I get:

File ~/Downloads/Meridian/meridian_env/lib/python3.10/site-packages/meridian/model/model.py:1863, in Meridian.sample_prior(self, n_draws, seed)
1854 def sample_prior(self, n_draws: int, seed: int | None = None):
1855 """Draws samples from the prior distributions.
1856
1857 Args:
(...)
1861 (https://github.com/tensorflow/probability/blob/main/PRNGS.md).
1862 """
-> 1863 prior_draws = self._sample_prior_fn(n_draws, seed=seed)
1864 # Create Arviz InferenceData for prior draws.
1865 prior_coords = self._create_inference_data_coords(1, n_draws)

File ~/Downloads/Meridian/meridian_env/lib/python3.10/site-packages/meridian/model/model.py:1820, in Meridian._sample_prior_fn(self, n_draws, seed)
1808 gamma_gc_dev = tfp.distributions.Sample(
1809 tfp.distributions.Normal(0, 1),
1810 [self.n_geos, self.n_controls],
1811 name=constants.GAMMA_GC_DEV,
1812 ).sample(**sample_kwargs)
1813 base_vars[constants.GAMMA_GC] = tfp.distributions.Deterministic(
1814 base_vars[constants.GAMMA_C][..., tf.newaxis, :]
1815 + base_vars[constants.XI_C][..., tf.newaxis, :] * gamma_gc_dev,
...
5981 def raise_from_not_ok_status(e, name) -> NoReturn:
5982 e.message += (" name: " + str(name if name is not None else ""))
-> 5983 raise core._status_to_exception(e) from None

InvalidArgumentError: cannot compute Mul as input #1(zero-based) was expected to be a double tensor but is a float tensor [Op:Mul] name: