Is it a necessity to have an append only database?

[RobStallion]

@nelsonic I am wondering what your thoughts were on this app being built without an append only log (alog) database.

• Is the functionality crucial to the application?
• Can the alog functionality be added in at a later date?
• Are there any alternatives that we could use instead of alog, like taking very regular back ups of the database for example?

[nelsonic]

@RobStallion thank you for opening/capturing this issue/question 👍
I have asked myself this and pondered it extensively ... 💭
and the discussion/decision must be captured so that others can read + refer to it. 📝

Short Answer: No.

The short answer to the question "Do we need to have X before we can launch?"
where X is almost any technology or feature is almost always "No".
Most features/functionality and/or technology is not required to launch/test an idea.
The beauty of "agile" is that we can build an MVP, test it and iterate based on "user" feedback.

What is our "USP"?

We could build a basic Todo List with integrated Time Estimation and Tracking functionality without the ability to undo or see the complete history of a task. We could build the app without an append-only-log and ship it to people. But what would our USP be? Why would it be obviously better than "Toggl"?
(which we all agree is a "meh" UX ...) What would compel a person who does not currently track their tasks/activities to use our App instead of the many others out there?
There are literally hundreds of Todo List and Time Tracking apps available.
(there weren't as many in 2015 when we first started building "Time" ... but that's another story ... 🙄)
The point is what is the differentiator from a UI/UX perspective?

If we use a "standard" Ecto/Postgres CRUD approach then a record will have an inserted_at and updated_at timestamp. This might sound like "enough" when it comes to a simple todo list item, where you start and finish the task in one go, but the moment we have anything more realistic like a task that you have to start, put down, and resume later, the single updated_at becomes useless. What we need is a way of starting and stopping a task multiple times and being able to track each time block. And then being able to see the complete history of the task simply by replaying the "log".

We could achieve this by simply creating a new "timer" record each time a timer is started and ending that timer record when it gets stopped. And that would work fine on the surface. But what happens if someone wants to manipulate the data? What if someone artificially changes all the start & end times on their timers to inflate the amount of time they spent on a task. The person reviewing the task (let's call them the task "owner") will be none the wiser that (a) the person executing/performing the task always "forgets" to "start" the timer before they start the task and (b) they stop the timer and then manually dial the clock to a later end-time to make it look like they spent more time on the task ...
In personal time-tracking app this doesn't matter. But in an app where you want to help people be accountable it matters immensely.
I'm not saying that it's a "crime" to forget to start a timer (in fact applying a different start time to a timer is a necessary feature that we will add in due corse), what I am saying is that not having a record of how many times a person has changed the start/end time is the problem ...
I don't know if I'm being clear here ... 🤔
Basically: by having an append-only log where all changes are recorded there is never any doubt as to when data/history was altered and anyone reviewing both their own history or that of a co-worker can easily see if they are being "forgetful" with their start/end times.

To be clear: for @dwyl I still think that we should not be paying anyone by the hour/day.
I think @dwyl should be a "ROWE" where everyone is payed based on the Result/Outcome not the time it took them. This is a difficult balancing act in practice because it makes estimation much more difficult; we go back to the "T-Shirt Size" estimation which is useless for planning a project.

"As far as the customer is concerned, the interface is the product." ~ Jef Raskin

Our objective is to make the interface (UI/UX) between the person and their "list" as seamless as possible.

Longer Answer: Yes, 100%; "Undo" is Core UX

I consider the "undo", "replay task" and "task history" UX to be so central/core to the functionality we are building that I feel it's worth investing the time up-front to get https://github.com/dwyl/alog working exactly how we want it.
It's not just a "nice to have" it's a "killer feature" to have complete history (audit/accountability) in a system.

UX is Everything

At launch in July 2007, the original iPhone did not have many of the most useful features we take for granted today: GPS, copy-paste, basic text editing (you could not scroll to a point in text and edit from a cursor. if you wanted to change text you had to delete all text after it and re-type...!), Notifications, no "push" email (which BlackBerry had perfected by the time), a user-accessible file system so no files could be downloded, MMS (which was the standard for sending media/photos at the time) ... etc. The AppStore was only launched in June 2008 (so no 3rd party apps for first year).
None of that mattered to early adopters who lined up around the block to buy the "unfinished" device.
The point is that the "core" functionality for the iPhone worked. Calling, SMS, (POP) email, music playing, basic camera, multi-tasking (switch between music and call) "full" web browser.
The original iPhone was an MVP in terms of features, but it was way ahead of its' time in terms of UX.
The multi-touch glass screen which was a "blank page" allowed infinite possibilities/creativity.
The on-screen keyboard could change based on language preferences which meant that one device could work in any country/language this was instrumental
see: https://en.wikipedia.org/wiki/IPhone_(1st_generation) + imore.com/history-iphone-original
The reason I'm bringing up the iPhone as an example in the context of the App we are building is simple: UX is everything. The underlying UX that enabled the App revolution was the multi-touch glass screen. The "core" UX in the App we are building is how the data is stored for each record.
Having a time-series (or append-only log) means we can build some incredible UX that will blow people's minds with history/timelines, etc.

Perhaps a better answer to this question would be to show you the UX in the form of a Wireframe.
Which means I need to first fill in the gaps in this issue: dwyl/learn-wireframing#1 💭

Until I am able to write out what is in my head, please guide yourself based on dwyl/technology-stack#67
All our apps should have logging and analytics built-in from Day 0.

[nelsonic]

@RobStallion following on from your question "Do we need to have an append-only database?" ❓

I've opened the following SPIKE issue dwyl/ecto-postgres-pubsub-spike#1 to investigate if we can have a PostgreSQL Pub/Sub LISTEN function that listens for the insert and update event on a table and automatically inserts the record into a "history" table. 📜

If the SPIKE is successful, we won't need to have an append-only database and can instead use the "standard" Ecto CRUD and still have a complete history of all data thus allowing "undo" UX/UI. 💡