Question: Best strategy to migrate models from MongoDB ODM to something that supports DynamoDB for persistence

[blockjon]

I'm looking into migrating some of my models (which today are stored with Doctrine) from doing persistence in MongoDB (using modgodb-odm) into DynamoDB instead.

I'm looking for the path of least resistance to achieve this. I am seeking feedback/guidance from the DoctrineODM maintainers as to what would be my best path forward? I noticed the doctrine key/value store project ... was uncertain if it was the best fit for exactly what we want.

Ideally we would have both a fluent query builder and also a document manager.

One of my colleagues has suggested that I write a new codebase which is specifically an ODM just for DynamoDB from scratch.

All options are on the table. Please advise.
@malarzm @jmikola @alcaeus

[malarzm]

One of my colleagues has suggested that I write a new codebase which is specifically an ODM just for DynamoDB from scratch.

This reminds me of #1209. I still think that it'd be cool to make the ODM work with more databases than just Mongo and I still think we lack resources to do so HOWEVER we have great opportunity right now as we need to rework half od ODM for new driver anyways and we could at least look into this.

But frankly as much as I'd love to make this happen I don't think it will anytime soon given we all have dayjobs and, at least I, are working on ODM in my free time mostly and can't really commit more.

[eerison]

Hey @malarzm

as you have such experience with mongodb and I believe you passed for many issues in the past!

When you have sometime, could you post here tips or concerns what should be the "best" approach to support DynamoDB

I don't know if already exist a "core" repository that contains the basic code that is used for NoSql database, and maybe we could create different repositories and plug into this "core" like adapters.

well if you could help somehow with your experience it will be really helpful!

[malarzm]

I don't know if already exist a "core" repository that contains the basic code that is used for NoSql database, and maybe we could create different repositories and plug into this "core" like adapters.

Sadly (or luckily, depends on one's point of view :) ) we did not end with pluggable adapters nor interchangeable platforms like DBAL, in fact we're coupled with MongoDB's own library. To see the journey of going from old doctrine/mongodb to new implementation you could take a look at #1553 - this will give you a hint on what needed to be changed to have another library be responsible for connections.

When you have sometime, could you post here tips or concerns what should be the "best" approach to support DynamoDB

I'm afraid I don't have any golden tips as I have never worked with DynamoDB and I have no clue where and how it differs from MongoDB. Probably the best approach would be what doctrine/dbal does for ORM BUT that requires deep insight into differences between MongoDB and DynamoDB. And, on top of that, massive work to put that abstraction layer into ODM. Also the biggest con then is that ODM needs provide features that will be useful for all platforms, not just one. Another thing that gets immensely complicated will be queries. I mean SQL has its flavours but it pretty much looks the same everywhere: JOIN here, GROUP BY there. NoSQL databases have no common standards as far as I know 🤐

So after writing the paragraph above the best approach would be a separate project that could move at its own pace and adapt to DynamoDB's limitations/greatness :)

[alcaeus]

Disclaimer: speaking from my position as maintainer, not as an employee of MongoDB.

I have strong reservations about this, mainly because of my experience with ORM. Due to its support for multiple databases, there's a "lowest common denominator" approach to DBAL. For example, adding support for features like complex types in PostgreSQL is rather complicated, as other database systems don't support this kind of stuff, so portability becomes an issue. Same goes for queries; made worse by the fact that those databases pretend to share a common query language (SQL) but with several differences and specialties. This makes maintaining the DBAL rather painful, as you need to keep other database systems in mind when making changes.

Contrast this with the ODM, where we only support a single database, so we don't need to worry about what would happen if people connect to a different database. I'd be very sad if we were to give up that advantage, as it would take up maintenance time which we honestly don't have.

As for rolling this yourself, I believe your best bet is to start using something like the Doctrine Skeleton Mapper, which takes care of the management part but leaves persistence open for you to implement. This can be used to persist data to DynamoDB, while still having the object management part taken care for you. It's still a lot of work, but IMO considerably less than making ODM work with DynamoDB in addition to MongoDB.

[eerison]

Thank you @malarzm and @alcaeus for you time, I really appreciate that, I will take a look into the repositories that both of you suggested and understand better how it works.

when I have something started, I would like to post here to get your opinion again. But fell that I have good points from where start ❤️ !