Zygote context cache incorrectly(?) persists between AD calls

[marius311]

You can see below more objects accumulate into it every call. I'm not an expert but @oxinabox on Slack suggested this might be incorrect:

julia> let
           ad = AD.ZygoteBackend()
           AD.derivative(ad, exp, 1.)
           display(ad.ruleconfig.context.cache)
           AD.derivative(ad, exp, 1.)
           display(ad.ruleconfig.context.cache)
       end
IdDict{Any, Any} with 3 entries:
  Box(true)    => RefValue{Any}((contents = nothing,))
  Box(2.71828) => RefValue{Any}((contents = nothing,))
  Box(2.71828) => RefValue{Any}((contents = nothing,))
IdDict{Any, Any} with 6 entries:
  Box(2.71828) => RefValue{Any}((contents = nothing,))
  Box(2.71828) => RefValue{Any}((contents = nothing,))
  Box(true)    => RefValue{Any}((contents = nothing,))
  Box(2.71828) => RefValue{Any}((contents = nothing,))
  Box(true)    => RefValue{Any}((contents = nothing,))
  Box(2.71828) => RefValue{Any}((contents = nothing,))

I have a non-MWE where this seems to cause an error, but the error is gone if I manually wipe it. Not sure if it could also cause subtly wrong results. Perhaps it should be wiped before or after each pullback call?