App Crashes after Opening The Economist Podcasts Page

[ezebecke]

From Igotdes:

User in #5187327-zen reports that their app crashes daily when they open the page for The Economist Podcasts (https://nodeweb.pocketcasts.com/admin/podcasts/558366).

App version : 7.19.2 (7566)
Phone: Google - Pixel 4a (5G) - Pixel 4a (5G) - bramble
Kernel: 4.19.220-gadc60de00e3a-ab8172795 - abfarm-01373
Android version: 12 SDK 32
They have tried:

deleting all episodes from The Economist Podcasts
unsubscribing from The Economist Podcasts, restarting the app, and resubscribing
App logs show:

I 02/5 12:03:43 Observing podcast 262069e0-e7df-0132-0ef7-059c869cc4eb episode changes
E 02/5 12:03:43 Fatal crash.
java.lang.IllegalStateException: Couldn't read row 562, col 4 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
User acknowledges the bug is minor, but would appreciate any support we can give.

[ezebecke]

From geekygecko

Here is some notes on the issue. Not anything to let the user know yet.

This issue seems to be related to reading too much from the database at once.

Here is a similar Google Issue.

seems like the list is really big where it is not fitting into a single CursorWindow. You can annotate the live data returning method with @transaction so that it will block the database while reading.
it will slow down your parallel access to database but that is the only way to do multi-cursor consistent read.

This is also the 4th most common crash in Firebase.

From a crash in Firebase it seems to be coming from the following method in the EpisodeDao.

@Query("SELECT * FROM episodes WHERE podcast_id = :podcastUuid ORDER BY published_date DESC")
abstract fun observeByPodcastOrderPublishedDateDesc(podcastUuid: String): Flowable<List<Episode>>

This is only used on the podcast page to monitor all the episodes. PodcastViewModel

Fix ideas
We could try using the Paging Library to reduce the amount of data we are querying at once. The only issue is this might be tricky as we current doing processing over all the episodes in code, such as sorting by groups like seasons.

[ezebecke]

From rustyshelf

"episode_count":2500
I feel like paging might not be such a bad idea considering the volume of episodes that is. Since other episode lists like filters are hard capped at much smaller numbers, the podcast page might be the only place we need this?

[ezebecke]

From geekygecko:

https://issuetracker.google.com/issues/65045157

you can also annotate the query with @transaction if you really want to fetch such a big data set.

There is another option to try @transaction but paging sounds like a better option for older devices.

[ezebecke]

From rustyshelf

If it happens every morning, then it's likely the podcast page is getting updated while someone is scrolling, causing the subsequent cursor fetch to be at the wrong index? I guess Room doesn't handle that? Is that what this Google Engineer is saying?

When reading data w/ the Cursor class, the first time it reads, it only loads 2MB of data. When that is full (as we read more rows), it re-runs the same query, skips first N rows and reloads. Between these 2 runs, if data changes, cursor may return wrong value.

So using @transaction bypasses that 2MB limit and runs the query every time you scroll or something? Might be interesting to test that first, since most of the phones this crashes on are fairly new. 84% Google Devices, 81% on Android 12.

[ezebecke]

From geekygecko

It looks like @transaction doesn't let anything else run until the all the queries have finished which might have other effects as the database is now locked. Paging does seem like the right solution for this if it's possible.

[ezebecke]

From rustyshelf

Will paging solve the scrolling while data is changing issue? From what I can tell every one of those cursor numbers is in the high hundreds, suggesting what has happened is that there aren't actually that many rows anymore. For example:

You open the page, there are 630 episodes
You scroll, the app starts requesting 50 -> 100, 100 -> 150
As part of a JSON update, an old episode is deleted. There are now 629 episodes
You scroll to the bottom, the cursor requests 600 -> 630, but when it goes to read episode 630, there's nothing there anymore and it crashes.
eg: I'm not sure whether this is true

This issue seems to be related to reading too much from the database at once.

[ezebecke]

From geekygecko

I'm not sure whether this is true

Yes it does seem more like the steps you have indicated. I would think their paging library would handle being at one page for awhile and then the data has changed either before or after it. It is their recommended library for handling large local dataset. But guess we wouldn't know until we tried it.

[ezebecke]

From thabotswana:

Another report in 5221363-zen

I opened the "Economist" listings. Without selecting anything Pocket cast shuts down and dumps me back to my home screen. If I restart Pocket cast it allows me back into the "Economist" section and behaves normally. It doesn't crash every time but it only happens with the "Economist" section.

[ezebecke]

Another report on 5384164-zd

The Pocketcasts app crashes, pretty reliably, when I select "The Economist" podcast from my main Podcasts screen after having not visited that feed for a while. By 'crashes,' I mean any playback in progress stops and the pocket cast app closes, revealing my Android home screen. Android does not prompt me to send a crash report.
If I restart the app, I can immediately view the economist feed without issue. I infer that perhaps the crash happens only if the feed has updated since my last visit.

[thabotswana]

Another report in https://woothemes.zendesk.com/agent/tickets/5563996

I am able to consistently cause pocketcasts to exit:

1. Open pocketcasts
2. Click profile in bottom right corner
3. Click downloads
4. Swipe left on any entry
5. Click the archive icon that was revealed.

For me, this force closes pocket casts.

[MiSikora]

This should be fixed by #1589

We should still migrate fetching our data in a paging manner but this is a bigger task and probably a project on its own.