Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support reactive streams #1

Closed
10 tasks done
aurelijusb opened this issue Apr 19, 2016 · 1 comment
Closed
10 tasks done

Support reactive streams #1

aurelijusb opened this issue Apr 19, 2016 · 1 comment
Assignees
@aurelijusb
Labels
enhancement
Milestone
0.4 Reactive streams

Comments

@aurelijusb
Copy link
Member

aurelijusb commented Apr 19, 2016 •
edited
Loading

Issue just for better development process: so I could mention issue number in commits and other could see near future plans

Problem

Using akka actors is good for asynchronous operations, but it too verbose for developers.
Also using actor pattern, it becomes hard to understand data flow

Solution

Use akka streams for abstraction over HTTP/Request and responses

Expected result

I would like to have something like:

val new Scarango(context)
//...
Source.fromFuture[Documents](scarango.byExample("{a:{c:1}}"))
    .runForeach(println(_))
    .filter(raw => toProduct(raw).price < 5)
    .map(_.getId)
    .map(scarango.Remove)
  • Getting results using Source notation
  • Writing to database using Flow notation (so errors or success could be dealt with)

Final goal

I could work with data flow like a stream:

  • Store to database, while still receiving from stow connection
  • Start responding to user, while not all elements are fetched from database (is it possible from ArangoDB side? - not yet - need stream based JSON parser)

Tasks

  • GetVersion: infrastructure to get request using reactive streams
  • CreateDatabase: infrastructure to store to database
  • GetAllDocuments: to test responding to user, while still reading from database
  • StoreRawDocument: to test back-pressure from database to initial client
  • CRUD for Collections: to have self contained tests
  • CRUD for Database: to have fully functional tests (for test _system database could be used)
  • Other functional compatibility to meet v0.2.4
  • Solution/Tests for error handling
  • Documentation, code quality: readiness for end-usage
  • Prepare example documents
@aurelijusb aurelijusb added this to the 0.4 Reactive streams milestone Apr 19, 2016
@aurelijusb aurelijusb self-assigned this Apr 19, 2016
aurelijusb added a commit that referenced this issue Apr 20, 2016
@aurelijusb
#1 Switch to reactive streams: get database version
f5130e4 
Removed old code, as actor based driver will be changed with akka streams.
Simple implementation of getting ArangoDB version using Akka streams.
Updated to newest ArangoDB version (2.8.7)
aurelijusb added a commit that referenced this issue Apr 20, 2016
@aurelijusb
#1 Compatible only with Java 8
5cb53d3 
Because of Akka http Graph
aurelijusb added a commit that referenced this issue Apr 20, 2016
@aurelijusb
#1 Compatible only with Oracle Java 8
e61da66 
Because of Akka http Graph
Later other version should be retested
aurelijusb added a commit that referenced this issue Apr 22, 2016
@aurelijusb
#1 Allow user to use driver in high and low abstraction level
f4e08b7 
For simple cases, there should be on-liners
For common use, thre should be context of current database to ease developing.
For complex flow graph and optimisations, there should be independent parts.
aurelijusb added a commit that referenced this issue Apr 22, 2016
@aurelijusb
#1 Create new collection
4bc8ea2
aurelijusb added a commit that referenced this issue Apr 22, 2016
@aurelijusb
#1 Create new document
c784ffd
aurelijusb added a commit that referenced this issue Apr 22, 2016
@aurelijusb
#1 Simple read from collection
3445c60
aurelijusb added a commit that referenced this issue Apr 24, 2016
@aurelijusb
#1 Simulating: Fetch results asynchronously
bef6ef1 
AragnoDB returns whole data in JSON text, split by text size (not by document boundaries)
Spray-json does not support parsing as a stream
HttpResponse.entity ByteString is returned whole, while TCP packets are sent separately

Also, added mandatory document fields for examples, how to parse importand data into schema-less documents.
aurelijusb added a commit that referenced this issue Apr 24, 2016
@aurelijusb
#1 Simple create database
39bfdcd 
Not passing user data
aurelijusb added a commit that referenced this issue Apr 24, 2016
@aurelijusb
#1 List databases
5a3aed9 
Reformatted tests to better see API coverage
aurelijusb added a commit that referenced this issue Apr 24, 2016
@aurelijusb
#1 Delete database
3488d3f
aurelijusb added a commit that referenced this issue Apr 24, 2016
@aurelijusb
#1 Create collection inside specific database
86f53f4 
Also added more context information to test output.
Moved context parameters for easier copy of immutable values.
aurelijusb added a commit that referenced this issue Apr 24, 2016
@aurelijusb
#1 List collections in database
4b7e6c5 
Also reduced software code for unmarshaling and implicits.
aurelijusb added a commit that referenced this issue Apr 26, 2016
@aurelijusb
#1 Remove collection
15f53bd 
Also updated main documentation
aurelijusb added a commit that referenced this issue Apr 26, 2016
@aurelijusb
#1 Remove document
19430bf 
Refactored tests,
so smaller parts would be documented.
ScalaTest cases could be independent (e.g. run in parallel)
aurelijusb added a commit that referenced this issue May 1, 2016
@aurelijusb
#1 Get document
71d31c7 
Also includes example of converting custom document to Scala object.
aurelijusb added a commit that referenced this issue May 1, 2016
@aurelijusb
#1 Create database restricted to user
1b5fb51 
Also, updated documentation to better reflect ArangoDB API swagger format
aurelijusb added a commit that referenced this issue May 2, 2016
@aurelijusb
#1 Refactored: Separate classes for different abstraction levels
a1b3075 
So it will be less code for common uses (scarango.Results.version -> scarango.version).
Still allowing easy switch between abstraction levels (scarango.toStreams.toFutures.toAwait)

Cannot have abstract interface for ArangoDB API, as return types are different
and do not want hardcode whole flow as a type.
aurelijusb added a commit that referenced this issue May 2, 2016
@aurelijusb
#1 Handling errors: Minimal implementation
a350faa 
For Await - errors thrown as exceptions
For others - passed as Try[Result], so failed responses could be handled in separate stream.

Moved Request and Response function to objects, so IDEs could give more usable error traces.
aurelijusb added a commit that referenced this issue May 2, 2016
@aurelijusb
#1 Documentation: v0.3.1
92d38d4 
Dropping scarango-macros,
as it is not currently used and introduces compiler dependencies.
aurelijusb added a commit that referenced this issue May 2, 2016
@aurelijusb
#1 Improved publishing to Sonatype
31ece8f 
To publish:
See https://github.com/xerial/sbt-sonatype
publishSigned
sonatypeRelease
@aurelijusb aurelijusb mentioned this issue May 2, 2016
Merged
aurelijusb added a commit that referenced this issue May 2, 2016
@aurelijusb
Merge pull request #3 from Auginte/reactive-streams-#1
16592d5 
Reactive streams #1: Minimal functionality
aurelijusb added a commit that referenced this issue May 2, 2016
@aurelijusb
#1 Small documentation improvements
4f4b557
@aurelijusb
Copy link
Member Author

API coverage, improvements and bug fixes will be in separate issue.
Functionality implemented and released under https://github.com/Auginte/scarango/releases/tag/v0.3.1

@aurelijusb aurelijusb mentioned this issue May 3, 2016
Open
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aurelijusb aurelijusb

Labels
enhancement
Projects
None yet
Milestone
0.4 Reactive streams
Development

No branches or pull requests
1 participant
@aurelijusb