Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce civi.api4.authorizeRecord and civi.api4.validate #20533

Merged
merged 27 commits into from
Jun 9, 2021

Conversation

totten
Copy link
Member

@totten totten commented Jun 7, 2021

Overview

Combines prior PRs. Refactor so that events use more consistent style (ie civi.api4.{$TASK} with alias civi.api4.{$TASK}::{$ENTITY}). Description TODO.

A brief description of the pull request. Keep technical jargon to a minimum. Hyperlink relevant discussions.

Before

What is the old user-interface or technical-contract (as appropriate)?
For optimal clarity, include a concrete example such as a screenshot, GIF (LICEcap, SilentCast), or code-snippet.

After

What changed? What is new old user-interface or technical-contract?
For optimal clarity, include a concrete example such as a screenshot, GIF (LICEcap, SilentCast), or code-snippet.

Technical Details

If the PR involves technical details/changes/considerations which would not be manifest to a casual developer skimming the above sections, please describe the details here.

Comments

Anything else you would like the reviewer to note

totten and others added 19 commits June 6, 2021 20:18
The original form (`getBatchRecords()`) still returns an array of items.

The alternate form (`getBatchAction()`) returns an API call for fetching the
batchs - but this API call may be further refined (e.g.  selecting different
fields or data-pages).
…l pas

Context: There were three separate, concurrent PRs - two added more tests
and events to APIv4, and the third added a new entity (FinancialItem).
FinancialItem got merged first.  I'm working reconciling the other two...
and discovered that `FinancialItem` isn't passing.

Problem: When the `ConformanceTest` creates a `FinancialItem`, it doesn't
fill in valid values for `entity_table,entity_id`. These values are
important to the access-control criteria used in reading-back data.
This adds a static ::checkAccess function to all BAOs, which dispatches to
a protected _checkAccess function in that BAO, as well as a new hook:
hook_civicrm_checkAccess($entity, $action, $record, $contactID, &$granted)
Call checkAccess action before creating, updating or deleting
…hone, etc.)

Implements the _checkAccess BAO callback for contacts and the related entities
listed in _civicrm_api3_check_edit_permissions.

Switch APIv4 to stop using _civicrm_api3_check_edit_permissions
now that the checks are implemented in the BAO.

Also fixes a couple permission check functions to respect $userID variable.
The primary purpose of this is to provide a trait (`AuthorizedTrait`) to
describe the common semantics of of coarse-grained authorization check and
the upcoming fine-grained authorization check.

The extracted trait makes a few small changes:

* Change the default value from `FALSE` to `NULL`.  In grepping universe for
  consumers of `isAuthorized(0`, I could only find consumers that used
  bool-ish values.  So this should be the same for them.  However, for
  future cases, it will allow some distinction between NULL/FALSE.

* Use more type-hints. The type should be nullable-boolean.

* Mutators should be amenable to fluent style (e.g. `$event->authorize()->stopPropagation()`).
@civibot
Copy link

civibot bot commented Jun 7, 2021

(Standard links)

@civibot civibot bot added the master label Jun 7, 2021
@eileenmcnaughton
Copy link
Contributor

188 test fails eg
api\v4\Action\BasicActionsTest::testCrud
Civi\API\Exception\UnauthorizedException: ACL check failed

@totten totten force-pushed the master-api4-omnivent branch from a9e0712 to eccc78b Compare June 7, 2021 21:27
@totten
Copy link
Member Author

totten commented Jun 7, 2021

@eileenmcnaughton Yeah, I think that failure (which recurred in several places) was due to the (lack of) fallback behavior in scenarios where there was no BAO class. Several other failures were due to a copy-paste-typo that I made a few minutes before pushing up. Preliminary local test-run look good for a dozen relevant test-classes. Jenkins is now doing a new full run. 🤞

@eileenmcnaughton
Copy link
Contributor

hmm closer - but still 3 fails - this test should be handled in

function financialacls_civicrm_pre($op, $objectName, $id, &$params) {

but it seems the code has not yet been moved to the extension & is sitting here

if (!empty($params['check_permissions']) && CRM_Financial_BAO_FinancialType::isACLFinancialTypeStatus() &&

api_v3_FinancialTypeACLTest::testDeleteACLContribution
api call should have failed but it succeeded We expected a failure for Contribution delete but got a success: Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 1
[values] => Array
(
[1] => 1
)

)

Failed asserting that 0 matches expected 1.

/home/jenkins/bknix-dfl/build/core-20533-3e0yp/web/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php:86
/home/jenkins/bknix-dfl/build/core-20533-3e0yp/web/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php:137
/home/jenkins/bknix-dfl/build/core-20533-3e0yp/web/sites/all/modules/civicrm/tests/phpunit/api/v3/FinancialTypeACLTest.php:313
/home/jenkins/bknix-dfl/build/core-20533-3e0yp/web/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:237
/home/jenkins/bknix-dfl/extern/phpunit8/phpunit8.phar:671

@totten totten force-pushed the master-api4-omnivent branch from eccc78b to 3cb1873 Compare June 7, 2021 23:50
@totten
Copy link
Member Author

totten commented Jun 7, 2021

api_v3_FinancialTypeACLTest::testDeleteACLContribution

Yeah, Coleman had already put code for this in the extension. I changed it from a hook-listener to Symfony-listener, but the way I registered it was a little flaky. (It would only work during a solo test-run - not during a full test run.) Switching the registration mechanism has fixed this locally. (It now does $dispatcherDefn->addMethodCall('addListener',...).)

@eileenmcnaughton
Copy link
Contributor

@totten so in terms of moving that code out of v3 api into the extension nothing is changed by this ?

@totten
Copy link
Member Author

totten commented Jun 8, 2021

@eileenmcnaughton The incantations are changed, but the structure is the same as Coleman's prior PR. So civicrm_api3_contribution_delete() delegates from here a98791f#diff-e615c51bff0c1d7b67d60ac46cfe03e84261c8be20aff18907529796c25b48adR207).

   if (!empty($params['check_permissions']) && !\Civi\Api4\Utils\CoreUtil::checkAccessDelegated('Contribution', 'delete', ['id' => $contributionID], CRM_Core_Session::getLoggedInContactID())) {

which lands in financialacls.php (in an hook/event listener).

I suppose one might call it a quirk that an APIv3 implementation calls a utility from APIv4-land, though I don't think anything is lost in doing so. If it seems problematic for some reason, then we just need to rename \Civi\Api4\Utils\CoreUtil::checkAccessDelegated() and tweak to v3/v4. We would also want to rename the event from civi.api4.authorizeRecord to civi.api.authorizeRecord (with a footnote that it has very limited applicability to v3).

@eileenmcnaughton
Copy link
Contributor

@totten yeah - I guess my feeling is that line really shouldn't be in the v3 api - ie financialacls are supposed to being transformed into 'just another hook-implemented acl' - the change you cite makes it MORE generic - but it's still tied to contribution.api as opposed to 'any entity might do this'

(This discussion is non-blocking on the main effort)

@eileenmcnaughton
Copy link
Contributor

On acls - I think the financial permissions acls work the opposite way to the rest - ie they reduce other-wise given permissions - ug

totten added 6 commits June 7, 2021 21:10
Technically, there is an inheritable contract-change here - modifying
`isAuthorized()` to accept the current user ID. However, I grepped
universe for references:

```
[bknix-min:~/bknix/build/universe] grep -ri isAuthorized $( find -name Civi )
```

And all references were internal to `civicrm-core.git`.  This makes some
sense, given the available alternative extension-points
(`Civi\Api4\$ENTITY::permissions()` and `civi.api.authorize`).
…sDelegate.

Code paths:

* Before: There are many callers to `$bao::checkAccess()`.
* After: There is only one caller to `$bao::checkAccess()` (ie `CoreUtil`).

Delegation mechanics:

* Before: When delegating access-control to another entity, various things invoke  `$bao::checkAccess()`.
* After: When delegating access-control to another entity, various things invoke `CoreUtil::checkAccessDelegated()`
1. This removes the special-case where `CustomValue::checkAccess()` needs an extra parameter
   to identify the target entity.
2. This lines things up to do the swap from `_checkAccess()` to a hook/event listener
This change invovles a few things:

1. Pass the `AbstractAction $apiRequest` instead of the tuple `string $entity, string $action`.

2. There are a couple cases where we don't actually want to re-use the current `$apiRequest`.
   Switch these using `checkAccessDelegated()`.

3. Always resolve the userID before calling `checkAccessRecord()`. `$userID===null` can mean
   two different things (ie "active user" vs "anonymous user").  By
   resolving this once before we do any work with `checkAccess()`, we ensure that it will
   consistently mean "anonymous user" (even if there are multiple rounds of delegation).

3. Change the name from `checkAccess()` to `checkAccessRecord`. There are a few flavors of
   `...checkAccess...`, and this makes it easier to differentiate when skimming.
…e `$granted=NULL`.

Regarding invocations:

* Before: There are three different ways `Hook::checkAccess()` may be invoked, e.g.
    * `CRM_Core_DAO::checkAccess()`, which sprinkles in a call to `static::_checkAccess()` before `Hook::checkAccess()`
    * `CRM_Core_BAO_CustomValue::checkAccess()`, which sprinkles in a call to `checkAccessDelegated()` after `Hook::checkAccess()`
    * `CoreUtil::checkAccessRecord()`, which delegates to one of the above (if appropriate) or else calls `Hook::checkAccess()`
    * `CoreUtil::checkAccessRecord()` is the most general entry-point
* After: There is one way to invoke `Hook::checkAccess()`, and it incorporates some qausi/unofficial listeners.
    * `CoreUtil::checkAccessRecord()` is still the most general entry-point.
    * `CoreUtil::checkAccessRecord()` fires `Hook::checkAccess()` unconditionally
    * `CoreUtil::checkAccessRecord()` calls `CRM_Core_DAO::checkAccess()` and/or `CRM_Core_BAO_CustomValue::_checkAccess()`,
      which are now quasi/unofficial listeners for the hook

Regarding initialization and passing of `$granted`:

* Before: The value of `$granted` defaults to `TRUE`. Listeners may flip between `TRUE`/`FALSE`. The value of `$granted` is passed to each listener.
* After: The value of `$granted` defaults to `NULL`. Listeners may flip to `TRUE`/`FALSE`. If it remains `NULL` until the end, then it's treated as `TRUE`.
  The value of `$granted` is not passed to each listener.
* Comment: IMHO, this is an overall simplification.  If you pass in `$granted`, then each listener has to decide
  whether/how to mix the inputted value with its own decision.  (Ex: Should it be `return $grantedInput &&
  $myGrantedDecision` or `return $grantedInput || $myGrantedDecision` or `return $myGrantedDecision`? That choice appears to be
  carefully informed by the context of what steps ran before.) In the updated protocol, each `_checkAccess()` a smaller scope.
@totten totten force-pushed the master-api4-omnivent branch from 3cb1873 to af4cccf Compare June 8, 2021 04:10
@totten
Copy link
Member Author

totten commented Jun 8, 2021

a) No kidding 🤕

b) My only fear is that it's not so simple -- more pre-existing/unarticulated issues may intervene and prompt us to pivot to a different approach. (An alternative approach - which has been sketched by Coleman before - is to temporarily override the global contact ID - and then set it back.)

c) Right. It's that interplay between things like (i) access CiviEvent, access CiviMail, edit all contacts, etc (aka coarse-grained/guardian/civi.api.authorize/PermissionCheckSubscriber/AbstractAction::isAuthorized()) and (ii) ACLs, financial ACLs, etc (aka fine-grained/record-level/civi.api4.authorizeRecord/CoreUtil::checkAccessRecord()).

On acls - I think the financial permissions acls work the opposite way to the rest - ie they reduce other-wise given permissions - ug

Well... it passes any existing tests... FWIW, this revision can distinguish between 'explicitly authorized' ($authorized===TRUE), 'explicitly prohibited' ($authorized===FALSE), and 'indeterminate' ($authorized===NULL). So for financialacls, it does a prohibition via ($e->setAuthorized(FALSE);).

--

I was curious to see: When, if ever, does the system currently do permission-checks on behalf of other users (who are not presently interacting with the app)? Such use-cases ought to use CRM_Core_Permission::check($perm, $userID), and grepping for that in core+universe (Permission::check(.*,) found a few candidates. These cases seemed fairly legit:

  • Authx (ext): You can grant a permission like authenticate via password or authenticate via api key. At the moment when a user submits their password, they have not logged in yet, but you still need to check the corresponding permission.
  • ACL Cache/API: The signatures for CRM_ACL_API::whereClause() and CRM_Contact_BAO_Contact_Permission::cache() allow you to populate the ACL cache on behalf of a non-present user. I couldn't see a code-path which takes advantage of this today. (They all seemed to lean on getLoggedInContactID().), but maybe I just missed it. Even if it's not there today, it could make sense someday. Ex: If you implemented an ACL-cache based on background-updates, then it could loop through a bunch of non-present staff-users and try to update/recompute their ACLs.
  • Group Admin (ext): This hooks into ACLs. Since the ACL Cache/API allows lookups for non-present users, it has to follow suit.
  • Afform (ext): This hooks into Permission::check(). Since Permission::check() allows lookups for non-present users, it has to follow suit.

Maybe the ACL cache example is salient? Suppose you had a background-script proactively populating the ACL cache for staff-users - and the ACLs depended on smart-groups. The smart-groups are built from APIv4 queries. It would be important for all the permissions to be resolved. OTOH, for ACL cache, you'd probably want to just do the get() action as the target-user...

--

Speaking of ACLs - there are ACLs for CustomGroups. Is CRM_Core_BAO_CustomValue::_checkAccess() doing the right thing by delegating the access-check to Contact.update? Shouldn't it actually be consulting the CustomGroup ACLs? Or is that happening in some other way?

@eileenmcnaughton
Copy link
Contributor

Re customGroups - I think the acls have traditionally done this really weirdly - ie our favourity function 'getTree' sets permissions to true (often inappropriately) and returns metadata without acl-limited custom fields - I suspect that is behind some of the test fails in this PR #20531

My head still hurts - but I'm not sure if any of the edge cases invalidate the approach here

@colemanw
Copy link
Member

colemanw commented Jun 8, 2021

Eileen raises a good point about gatekeeper vs fine-grained checks - for most entities the latter overrides the former, which means we should not be doing gatekeeer checks in the api at all for any entity which supports acls.
This is already the case for Contacts, where api permissions are set to [] (empty array implying no permission needed).
My intention was to do the same for other entities one-by-one as we get acl checks working for them in the api, and I was hoping this pr would take care of a lot of them.
Note that the coarse-grained permission checks actually do happen for contacts as part of the fine-grained acl check, but the order is flipped and they are checked last to give acls a chance to override them.
Financial ACLs are indeed "ug" but the reverse-of-the-norm order thing might not be a big deal in the model being developed in this PR, because the _checkAccess function can implement any logic it wants, including flipping the order of checks to meet legacy expectations.
Re: CRM_Core_BAO_CustomValue::_checkAccess() - you're right @Eileen that function ought to be delegating to custom group acls not contact acls. Good catch.

@eileenmcnaughton
Copy link
Contributor

@colemanw @totten - I'm trying to parse ^^ into whether this PR is now mergeable / it's time to cut the rc?

…ture update.

Context: AuthorizeEvent did not allow tracking userID.  AuthorizeRecordEvent
is spec'd to track userID.  This is a step toward supporting checks when the
target user is non-present (ie not the user in the browser/session).
However, this step is not *sufficient* - additional work is also needed to
support non-present users.

Original: AuthorizeEvent and AbstractAction::isAuthorized did not report
current userID.  However, the wiring for AuthorizeRecordEvent is spec'd
to allow userID.

Previous: Made a breaking change in the signature of
AuthorizeEvent/AbstractAction::isAuthorized() to report userID.  However,
even with the break, it's not clear if this is the best approach.

Revised:
* Both AuthorizeEvent and AuthorizeRecordEvent report `userID`. This allows consumers to start using
  this information -- laying the groundwork for future changes.
* If an existing event-consumer ignores the `userID`, it will still work as correctly as before. This is
  because we guarantee that the userID matches the session-user.
* The signature of `AbstractAction::isAuthorized()` matches its original. No BC break. However, the method
  is flagged `@internal` to warn about the prospect of future changes.
* In the future, after we do more legwork on to ensure that the overall
  system makes sense, we may flip this and start doing non-present users.
@totten totten force-pushed the master-api4-omnivent branch from 45f7c0f to 70da392 Compare June 9, 2021 03:29
@totten
Copy link
Member Author

totten commented Jun 9, 2021

@colemanw I've pushed up both changes. I'll work on the description tomorrow - but it should be testable now.

@colemanw colemanw force-pushed the master-api4-omnivent branch from 6de9c39 to b87406e Compare June 9, 2021 18:23
Before: Reports that access is available based one delegated-check for `Contact.update`

After: Reports that access is available based on multiple checks:

  1. The user must have access to the relevant CustomGroup (by way of ACL or perms)
  2. The user must have acces to the underlying entity (by way of checkAccessDelgated)

Comments: I did a bit of testing with `Custom_*.get`, and it does seem to
give access to single-value CustomGroups. So I removed a comment about
multi-value CustomGroups and expanded to a larger list of entities.
@colemanw
Copy link
Member

colemanw commented Jun 9, 2021

This is looking good and is passing tests. Feels safe to merge at this point & we can hammer on it during the RC.

@eileenmcnaughton eileenmcnaughton changed the title (WIP) Introduce civi.api4.authorizeRecord and civi.api4.validate Introduce civi.api4.authorizeRecord and civi.api4.validate Jun 9, 2021
@eileenmcnaughton eileenmcnaughton merged commit 384f4ac into civicrm:master Jun 9, 2021
@eileenmcnaughton
Copy link
Contributor

@totten it is merged - let the great branching commence

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants