v7.13.0

7.13.0 - 2024-01-03

Just in time for the New Year!

A number of changes all wrapped up in one large PR #906 by @epugh. This one is for you @david-fisher ;-).

• Now when you judge a query doc pair for a Judgement Book we update in the background any associated Cases that have that query. This eliminates the need to manually refresh the ratings for a Case from the Judgement Book! Thanks @david-fisher for the suggestion in #829.

• Provide an option to allow queries that exist on a Book to be created in the Case if they are missing. The default behavior is to NOT create queries in a Case from the Book. Thanks to @jvia for this suggestion. #871.

• Band-aid the awkwardness that Cases have Queries/Ratings while Books have QueryDocPairs + Judgements, and that there are attributes on a Query like information need, notes, or options that don't exist on a QueryDocPair. So now we preserve them, which facilitates round tripping better. Upload a Book? Upload a Case? It doesn't matter ;-) Long term we need to merge these two data structures...

• When Books were first added to Quepid, they didn't follow the same permissions/sharing structure that Scores and Cases did. Now you can share a Book with multiple Teams! Fixes #907

• And as an aside, anyone can edit a custom scorer, not just the Owner of it. #762 suggested by @david-fisher.

v7.12.2

7.12.2 - 2023-12-01

We discovered that your custom headers can be very long! Thanks @tiagoshin for your patch. #903 by @tiagoshin.

7.12.1 - 2023-12-01

7.12.0 introduced a change in splainer-search that breaks searchapi :-(.

• Rollback splainer changes, but also some polish on the UX for going from the Case judgement screen to the Share with a Team. Also make parsing the csv file for the static api more robust. #894

7.12.0 - 2023-11-30

7.12.0 - 2023-11-30

A flurry of changes based on feedback from the community. This is basically a "improve" Quepid release.

Features

• Now show the last run on using friendly times, like "one hour ago". #892 by @epugh.

Improvements

• Restored in the homepage and the case listing the "last run on" and "last run by" information. Adding this had been causing performance issues, but hoping we worked this out. #891 by @epugh.

• Some general fixes to handling static defined search case and search endpoints. #886 by @epugh.

• Also, the usual upgrades to Rails, Node etc. We are now on Rails 7.1.3, though we still need to update some configuration settings from Rails 7.0 to 7.1.

Bugs

• Respect the proxy setting for a search end point in doing a snapshot compare lookup, and in linking to individual docs. #893 by @epugh. Thanks @david-fisher for finding this.

• We now support custom headers for search apis. Thanks @dacox for the fix in o19s/splainer-search#138.

• Made more robust the adding of queries for a static CSV case. #889 by @epugh. Thanks to @tiagoshin for identifying the issue.

v7.9.2 - Finally we can import and export Cases and Books cleanly

7.9.2 - 2023-09-20

You can now export and import Case's and Book's cleanly from one Quepid to another Quepid, with proper error handling. You can also use this to export a Case or Book, make a bunch of changes externally using other tools, and then bring it back in.

Get a dump file named case_25.json via:

python case_export.py --case_id=25 --root_url=https://app.quepid.com --access_token=YOUR_ACCESS_TOKEN

And bring it back in via:

python case_import.py --case_file=case_25.json --root_url=http://localhost:3000 --access_token=YOUR_ACCESS_TOKEN

More docs and some sample scripts coming soon.

#834 by @epugh.

v7.7.0 - a Dashboard style home page

7.7.0 - 2023-08-31

We finally have a brand new homepage! Quepid started as a tool for relevancy engineers only, but today we have human judges using Quepid to rate documents, Search Managers who are keeping an eye on the offline metrics, and of course in more complex setups, networking types who configure the connection between Quepid and the Search Engine. In the past, regardless of who you were, we dropped you right into a Case. If you had never created one before, well, you got shoved into the Case creation wizard, whether you wanted it or not. It made for a unpleasant first experience for anyone other than the hard core relevancy engineer.

The new dashboard is an attempt to change that UX. It's just a first cut, and honestly, it probably doesn't quite make anyone perfectly happy. I'm hoping that we can get a lot of good feedback and learn from it. I could imagine in the future that based on what you do, we surface information you care about. For example, a network engineer would want to know that the Quepid can talk to each Search Engine. A Search Manager would be more interested in insights and summary progress information. Human judges want to know "what is my next task to do". So please share feedback!

Features

• New home page (and sidebar!) in #681 is added via #684 by @epugh.

Improvements

• The feedback message after creating an empty book is now smart about if there are related cases or not, and gives better next steps instructions. #796 by @OkkeKlein fixed by #797 by @epugh. Thanks Okke!

• The Cases listing page shows your cases from oldest to newest. @atarora opened #708 for sortable columns. Implementing this didn't go well, so as an alternative, changed the sorting to be from newest to oldest, taking into account the case update time as well as the last time you viewed a specific case. #795 by @epugh. We may want to redo the entire Cases listing page at some point.... Hopefully this is better @atarora ;-).

• Reworked the messageing (and error handling) when you attempt to open a case that hasn't been shared with you (or doesn't exist!). We had been showing the "You need to reload Quepid in the HTTPS" type message, when actually it was a "Make sure this case has been shared with you!" situation. Also reduced the amount of error messages in the browser and the server in this situation by being smarter. Thanks @cmorley for surfacing this. #792 by @epugh fixed by #793.

• Thumb images can use a prefix_url, just like regular images. #790 by @OkkeKlein is fixed in #805 by @epugh.

• Using Nginx and have huge documents being snapshotted etc. New configuration suggested by @OkkeKlein allows Nginx to be happy. #804 by @epugh.

Bugs

• Drop a link in Slack to team 3, and you see details about case 3;-). #733 by @epugh fixed by #794 by @epugh to only do unfurling for cases.

• Ace code editor insists on loading two javascript files from specific paths in Quepid. Despite best effort, couldn't change this. So #793 just mounts those javascript where Ace wants it to shut up some very loud error messages in the browser console. If you can't beat'em, join'em.

• When judging ratings using the Book of Judgements, you could sometimes get a situation where you create a judgement, and then tried to create a new one, hitting a constraint. #809 by @epugh makes this more robust.

• Guess what? 7 GB out of our 9 GB production database is scores for cases being inserted OVER AND OVER again! The AngularJS app uses event emitting to signal when scoring is completed, this causes extra events to be triggered, causing extra PUT of the case scores. This shouldn't matter as we have logic on the server side. However, due to a bug, that logic doesn't actually work. The unit tests for the front end and the back end each independently validate the logic, it's just when you put them together it doesn't work. This didn't matter for years, but the new Dashboard surfaces scores ;-). Fixed in #807 by @epugh.

7.6.1: Round Trip Human Ratings!

Lots has happened since the last release notes, so doing one big combined one below!

7.6.1 - 2023-07-06

• Chased down bug with showing you the previous judgement on the Human Judgement page. #779 by @epugh.

7.6.0 - 2023-07-05

Big overhaul on the human rating side of things. We now roundtrip ratings that you have made from the core Case screen into your Book of Judgements when you populate it. (We are also now tracking who the last person was to rate a doc in the Case screen). You can now also merge ratings from multiple Book's into a new Book, which lets you reuse your judgements in new combinations.

Additionally, to encourage your human judges, a progress screen has been added every 20 judgements showing progress and a leaderboard.

• #778 by @epugh introduces some "fun" into the rating process ;-).

• #766 introduced the synthesizing of a Book, and fixes #765, track who rates in the main quepid ui, #763, link from list of books in judgements to the book itself, and #761, Combining books loses judgments but still produces scores.

• #760 changes Quepid to use floats for ratings and judgements, which opens the door to implicit judgements.

7.5.0 - 2023-06-15

Need to interact with Quepid API's from outside of Quepid? We now support Personal Access Tokens! From your Profile page you can generate your own token and then use that to authenticate against Quepid. For example, you can programatically load a judgement directly into Quepid:

curl -X POST http://localhost:3000/api/books/2/judgements/ -H 'Authorization: Bearer 4a82040bf1b2d255c63833cb59fa9275' -H 'Content-Type: application/json' -d '{
  "judgement": {
    "query_doc_pair_id": 201
    "rating": 1
  }
}'

#759 and #756 by @epugh.

In support of this, we added new API's for judgements and ratings in #757 by @epugh.

Bugs

• Exporting Ratings in CSV format was returning the same rating for every single rater, regardless of what they did! Fixed in by @epugh with special help from @grahamwren. I randomly sat next to him on plane flight home and we paired on it ;-).

• Looking up the wrong info need for a Query Doc Pair! Fixed in #755 by @epugh.

• Missed a relative path for the tries history visualization. Fixed in #754.

7.4.1 - 2023-06-09

Lots of small things from working with Quepid using templates with OpenSearch. Also, if you haven't used https://github.com/o19s/agent_q, it's been updated to work with Quepid 7.x line. I use it to automate pulling my relevance metrics daily from Quepid cases ;-).

Features

• Rework how we handle OS/ES templates in Quepid so that you don't need to append /template to the url, instead, use the existence of the {"id":"my-template"} in the query parameters to decide what URL to use. This simplifies life when you are using a mix of templates and not since you don't need to change the url constantly. This was mostly in the Splainer-Search v2.22, v2.22.1, and v2.22.2 releases. https://github.com/o19s/splainer-search/

• Now, under Explain Query modal, you can see the rendered template query for a specific query! #751 by @epugh.

Improvements

• Kind of in the weeds, but early database migrations in Quepid back in Rails 4 days didn't have a version spec. Rails 7 complains about this. #739 fixed by #746 by @epugh.

• Introduced a new environment variable QUEPID_CONSIDER_ALL_REQUESTS_LOCAL that lets you bubble up errors message in a production Rails environment, which is helpful in troubleshooting deployments when you can't see the logs. #750 by @epugh.

• Package Jupyterlite to not require external network calls. #721 by @epugh fixed by #728 by @mkr.

Bugs

• A three point custom scorer should support keyboard shortcuts. #738 by @epugh fixed by #752 by @depahelix2021.

• Missing Documents Modal doesn't work when an ES/OS Query Template is defined. #747 fixed by #753 by @epugh.

• Populating a book of judgements would blow up. Plus the data model allowed a single user to rate a query/doc pair multiple times, which was icky. #734 fixed by #745 by @epugh.

• Use relative paths everywhere so when Quepid is behind proxies it works properly. #754 by @epugh.

7.3.3 - 2023-05-30

• When populating a book of judgements, if the title field wasn't title, then it wouldn't show up with the correct name. #737 by @epugh fixes this.

7.3.1 - 2023-05-26

• Fixed some additional API paths for use with a nested Quepid context. #736 by @epugh. Follow up to #719.

7.3.3

7.3.3 - 2023-05-30

• When populating a book of judgements, if the title field wasn't title, then it wouldn't show up with the correct name. #737 by @epugh fixes this.

7.3.0

7.3.0 - 2023-05-23

Improvements

• Importing Snapshots was a feature from the dawn of Quepid, but @epugh had never used it till now. Originally you had to put into the csv file the id of the Case you want to import into, which is awkward and error prone. Now you can import a snapshot via the Import modal for a Case. #727 by @depahelix2021 fixes #724 by @epugh.

Bugs

• OpenSearch and Elasticsearch have a _source field that is nested JSON. Turns out we don't support stringifying a JSON object in detailed exports or Snapshots. #732 by @depahelix2021 fixes #730 by @epugh.

• With the logic for matching Quepid TLS to the search engine TLS happening only via frontend logic (introduced in #719), we found the User Experience really rough. #731 by @epugh reworks the UX to be clear about when you need to reload your Quepid.

• We ship Jupyterlite for data manipulation, but maybe you don't have public network access? #728 by @mkr fixes #721 by @epugh.

• Cloning a Case loses the advanced settings. #729 by @epugh fixes #725 opened by @jvia. Thanks @jvia for reporting this!

7.2.1

7.2.1 - 2023-05-12

Well, that didn't take long... Wanted to use the "Import Snapshot" function, and discovered that the "Export Snapshot" function was busted.

Features

• We've had the ability to mark a Case as "Public" for a while, but we only supported it for the visualization of tries screen. As a spike, trial allowing public access to /api/cases/CASE_ID.json and /api/cases/CASE_ID/snapshots/SNAPSHOT_ID.json end points. #723 by @epugh.

Bugs

• Fix Export Snapshot. #722 by @depahelix2021.

7.2.0

7.2.0 - 2023-05-11

Features

• Interested in Rank Biased Overlap as a more powerful version of the venerable Jaccard metric? We have it now as an example Jupter notebook. Thanks to Tito Sierra at the 2023 US Haystack for introducing me to this metric. Thanks to @mkr for creating the draft notebook.

• Provide link to Team from Book of Judgements. #718 by @epugh.

Improvements

• Lots of fixes and cleanups for nesting Quepid under another domain in #719. Credit to @frankcelia for figuring out that we were too smart by half in letting the Rails app redirect traffic to try and match Quepid TLS to the search engine's TLS. Pulling that out has simplified our logic immensely. #719 by @epugh and @frankcelia.

• We're now on Jupyterlite 0.1.0 from our relatively old 0.1.0b14, and have a build process that will let us stay up to date with this rapidly evolving project. #709 by @mkr.