Multi-feeds aggregation ?

[wa0x6e]

Is there a way to get latest activities for a given user, from a list of predefined feeds ?

Suppose I have a feed A, B and C. I would like to retrieve all activities for the user just from the feed A and C.

[kigster]

Can you clarify what you mean by feed A, B and C? How do you plan to configure / define them, and then use them?

[wa0x6e]

Suppose I have a feed named user_navigation, saving all the pages the user have visited. I also have a feed name user_session, for the user sign_in/login/logout activities.

These 2 feeds are separates, because the user_session feed is hidden to the normal user, and just shown for the admin. So normal user see feed user_navigation, and admin see both feeds.

If a user can have multiple feeds, I want to be able to view a given user activities across all the feeds.

[kigster]

I see, thanks for the explanation.

You have two options:

1. Merge items from two feeds manually by fetching all items from each feed and the doing the merge yourself (just add two aarrays and sort them by the time).

2. Put both session and navigation items into the same feed, but serialize each item in such a way that you can tell whether the item is the session or navigation. The you don't need to manually merge anything.

I can add a filter option to the feed so that you can fetch a filtered set of items from the feed.

[wa0x6e]

I have explored both options already, but I think the best options would be a ZUNION directly on redis side.

Option 1 is viable only when the number of feeds is small. When there is 3 feeds, it requires loading 3 feeds with 50 items each (total 150 items), sorted and filtered down to 50. A lot of overhead.

Option 2 requires more overhead to extract items from a specific feed, as we must loop through each page to fill the minimum number of items for the pagination.

A ZUNION, with the same API as multi-user would be perfect.

[kigster]

While it's an interesting use-case, the gem was written with the idea that separate feeds are just that — separate. In fact, they might even be in completely different redis instances. Similar thing happens when sharding by user_id using twemproxy.

So the use-case of merging two separate feeds is not something I am interested in supporting.

However, what you are really asking is an effective way of filtering data within a single feed. That I can get behind.

K

[wa0x6e]

I see, thanks