[Accepted with Revisions] SDL 0273 - WebEngine Projection mode

[theresalech]

Hello SDL community,

The review of the revised "SDL 0273 - WebEngine Projection mode" proposal begins now and runs through May 19, 2020. Previous reviews of this proposal occurred January 15 - February 25, 2020, and March 24 - April 28, 2020. The proposal is available here:

https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0273-webengine-projection-mode.md

Reviews are an important part of the SDL evolution process. All reviews should be sent to the associated Github issue at:

#912

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of SDL. When writing your review, here are some questions you might want to answer in your review:

• Is the problem being addressed significant enough to warrant a change to SDL?
• Does this proposal fit well with the feel and direction of SDL?
• If you have used competitors with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
  Please state explicitly whether you believe that the proposal should be accepted into SDL.

More information about the SDL evolution process is available at

https://github.com/smartdevicelink/sdl_evolution/blob/master/process.md

Thank you,
Theresa Lech

Program Manager - Livio
theresa@livio.io

[Jack-Byrne]

1. This statement makes it sound like this should be the behavior for all apps that use PROJECTION should be activated into a web page?

The app HMI type PROJECTION should be enabled for in-vehicle apps. When apps with this HMI type are activated, the HMI should make the web page of this app visible on the screen. This web page will become the main window of the application.

2. I don't really understand the need for restricting Webengine projection apps to not be able to send show requests.

3. Was there an issue with creating a new app hmi type specifically for webengine projection apps instead of reusing type PROJECTION? I apologize if it has already been discussed.

4. Has there been any discussion related to what existing SDL features these webengine projection apps will actually find useful?

5. As discussed in previous issue threads, this proposal makes some pretty fundamental changes to SDL's principles. I wish there was more of a story/justification in the proposal for why the SDL project would benefit from implementing this feature.

[ghost]

Hi @JackLivio. Thanks for you review.

1. Do you mean "it sounds like this should be the behavior for all apps" like apps from any platform? The proposal is related only to applications running on the IVI in the WebEngine as proposed in SDL-0240. Just like SDL-0031 apps with permission to use PROJECTION should present their own UI.

2. The restriction is only related to the app's main window which is not visible at any time and technically it wouldn't exist. Show would still work for any other window like widgets. Successfully replying to Show isn't of a correct behavior because the addressed window just doesn't exist. Instead Core could return without executing the request. If there are use cases where projection apps can still show the main window, I'm happy to revisit this item.

3. No worries. No, this wasn't already discussed. The first version of this proposal was proposing a new variant but after reviewing I realized the similarities to the PROJECTION mode which allows apps to present their content. The difference is the used technology behind the scenes. Instead of streaming video we would use the website and the html DOM.

4. When we planned and designed the feature we discussed this point and we believe there are plenty of SDL features that are useful. The most important items are app policy management and app lifecycle. We would like to streamline the implementation to manage activating apps, deactivating or closing apps or run apps in background. Besides the benefits for us implementing projection apps, it's also beneficial for the projection app developer to use all the SDL features like app services, remote control, vehicle data widgets and many more.

5. In fact the proposal doesn't make any change to the SDL principles that don't already exist. The same rules apply to webengine projection apps as for mobile projection apps as described in the accepted proposal SDL-0031. The only difference is the implementation to project (no video streaming but show the app's webpage).

[Jack-Byrne]

Do you mean "it sounds like this should be the behavior for all apps" like apps from any platform?

I know you meant only for web engine apps but I just wanted to point out how someone might find it ambiguous.

2. I think when we create these restrictions we end up complicating things in the future when we do finally create a use case. Maybe an OEM wants to create a hybrid view that dedicates part of the main window to traditional templates and part of the window displays the webengine?

Also the HMI implementation could choose to respond with failures for show requests for webengine apps. I don't think its a necessary requirement for Core.

3. What if we create a new app hmi type, "WEBENGINE_PROJECTION"? Open to other naming.

FYI I believe there are specific rules/exception related to the hmi state & level for when the app is a PROJECTION application, similar to how NAVIGATION apps have special rules. Do we want those same rules to apply to webengine apps?

4. Ok thank you. I think this was a good response and should even be included in the proposal. Relating to 5, I know what content goes into the motivation section of the proposal is subjective, but I think that section could use some extra material. I find it helps tell the story and vision for new features.

[ghost]

1. Got it. I can modify the proposal and state following:

The app HMI type PROJECTION should be enabled for in-vehicle apps. When in-vehicle apps with this HMI type are activated, the HMI should make the web page of this app visible on the screen. This web page will become the main window of the application.

2. I get your point and I would support Show requests when we have demand of such a use case. I also thought of a hybrid use case but we decided against it as we need to focus on the requested set of features. Also we want to stay efficient with the given time and budget. If another SDLC member would like this hybrid feature included we're more than happy for contributions. Please note that such a hybrid feature should be agnostic to other projection modes (mobile nav and mobile projection), so it could make sense to propose this feature separately. Just so much to the hybrid idea: With this feature it would be even more important to provide capability information to the app and properly reply to Show requests. Otherwise apps would not know if the template main window exist. Sending RESOURCE_NOT_AVAILABLE in projection only makes sense telling the app that the main window is not available. Also window capabilities are in place to provide the capability information.

3. I'm not sure if adding a new HMI type that behaves the same as PROJECTION is a good idea. the HMI type would be app platform specific and I would prefer a more unified variant. Core can distinguish between a mobile projection app and a non-mobile projection app at least by the (secondary) transport type but also by device info and other sources.

You're right. NAVIGATION is a special HMI type and there are special rules in the scenario of multiple nav apps. This proposal only describes PROJECTION. Another proposal for WebEngine based nav apps is in the backlog and dependent of this proposal.

4. Yes I'm more than happy to add my comment to the motivation explaining the benefits of SDL.

[joeygrover]

I apologize for my late comment here. I really wanted to get a good understanding and make meaningful comments.

6.

With the current depth of this proposal, the HMI type should be used by 1st party OEM apps only. With future proposals and workshops the SDLC could open the HMI type to 3rd party by creating and defining proper driver distraction and user interface guidelines.

I do believe OEMs should be able to use all the features of SDL, like this proposal, as they wish when it comes to their own apps. What they do for their own apps does not affect other OEMs implementations of SDL (in most cases). What I do worry about is the effect a proposal like this will have on the ecosystem for third party apps. While there's an argument that this is operating in the same way the SDL-0031 works, I believe that is not completely accurate.

With this being for locally stored applications, there is not as much of an adverse effect to circumvent the SDLC app certification and could therefore be implemented before the SDLC agrees upon the guidelines for these types of apps. This would put any OEM who is following our path to a disadvantage compared to the OEM(s) who have chosen to ignore it. We simply do not have a way to effectively monitor and enforce this. Maybe the SDLC members do not feel this is a concern though. I can imagine Ford doesn't, but would like to hear from other members specifically.

7.

To the author's knowledge there is no impact to existing code unless there are barriers implemented in SDL Core preventing an in-vehicle app from registering with the PROJECTION HMI type.

I think there could be a big impact to the screen managers that will be created for the JavaScript Suite. Regardless of whether this proposal is accepted or not, the screen manager will need to be implemented to support templated implementations so how that manager works when there is no main window will have to be addressed.

8.

There are no alternatives available that the author feels competitive to the projection mode using a WebView.

I think there are some other items that could enhance the SDL experience without adding this feature:

1. Better user interfaces from OEMs. Most seem to be designed by engineers or lawyers. App partners should be involved in designing templates to understand what they want to see.
2. Create standard HMI designs and reduce the extra work the developer has to handle at run time for different UIs for each IVI.
3. Implement all UI features including template colors.

These would help create a better SDL experience for applications from all platforms instead of just one, which is more competitive in its own right. These actions help SDL fall more in line with what competitors are doing, but SDL could do better.

9. I would like to see how this changes the previously supplied flows for web engine apps that simply start a new window but returns to the SDL HMI. This would instead keep the opened web engine instance in focus, so the flow would change. I'm assuming just using the HMI type of PROJECTION the HMI would know what to do.

[theresalech]

The Steering Committee voted to keep this proposal in review, to allow the author time to respond to the comments, and for the Project Maintainer and SDLC members to continue their review and discussion on the review issue. It was urged that all SDLC members, especially OEMs, review this proposal and understand how it will impact their current and planned future implementations of SDL.

[ghost]

@joeygrover

7. At the time of writing this comment, the javascript managers are not part of any accepted proposal. The proposal that adds the screen manager to the javascript suite should consider reading the window capabilities and behave accordingly. This behavior should be considered for the screen manager independently of this proposal.

8. I think I was referring to "what other projection alternatives exist besides video streaming". With regards to improve SDL templates I agree to your list and I would append a few more items. This proposal is only about suggesting to use a different presentation technology than video streaming for projection apps addressing the concern of how to realize SDL-0031 for webengine apps.

9. The HMI would know what to do when apps are activated with ROJECTION hmi type. It should know from the device info that the application is a locally installed webengine app. Actually the the policy app id should be more than enough as the HMI installed the application before and send "AddAppProperties" to Core with the policy app id.

[joeygrover]

7. At the time this is true, and I was hoping the author could help provide their insight on how it work work even still. However, I suppose another piece of this is that the entire multiple window system and widgets manager update proposals have never been submitted. So maybe that should be a priority which will include such changes. Was this something the author or another person from their company could handle?

8. The reason I bring it up is you stated there is no alternative that supports the level of advantages as this feature. This is the piece I disagree with and why I brought up those points. The reasoning behind this type of UI is to provide a better UX and simpler SDL integration, but as stated this could be improved in other ways that are not as brute force of a change and all SDK platforms would benefit from. I do not see the author agreeing with me on this, so the point is more for the SDLC; no further discussion is necessary here.

9. I can understand what triggers the behavior, but I would like to know how it affects the previously supplied flows for web engines applications. Can you please provide such flows that are updated to reflect such an application?

[ghost]

7. Let's not digress. SDL-0216 introduced the window capabilities and the default "main window" which this proposal suggest to be the projection window. The proposal states The window capabilities of this open main window will be empty except for the window ID and physical button capabilities. which has effect to the screen manager as it's already reading the capabilities. If you take a closer look to the subclasses of the screenmanager in sdl_ios and sdl_java_suite you will see that the sdl libraries are already taking window capabilities into account. There is no projection mode specific behavior required for the screen managers.

I think the second point of Jack's comment is closely related to this point as he describes the scenario if an app can be PROJECTION but also have a template main window (some sort of hybrid mode). If we are more specific, this hybrid mode would require modifications to the definition of the predefined default main window. I think the easiest approach would be to introduce a new projection window type and a predefined main projection window which will be added to the window capabilities array if the app is a projection app (including mobile navigation apps). Still, the default main window may be empty when not supporting text, buttons and graphics if the HMI isn't including a templated main window. Alternatively it could be omitted from the window list but I'm afraid the libraries could crash as the expect the default main window to always exist.

I realize that the below statement from the proposal isn't really valid as it's in the scope of the HMI. Core should not overrule the decision of the HMI.

The window capabilities of this open main window will be empty except for the window ID and physical button capabilities. Show requests that address the main window won't have any effect on the HMI. If the app sends this request, Core should return an unsuccessful response with the result code RESOURCE_NOT_AVAILABLE.

I hope my insight was helpful. If you have any further questions or feedback to the suggested window capability change please don't hesitate to send me your feedback. I'm more than happy to help.

8. I am sorry but if we are not having a discussion to your point please allow me to provide my comment for the SDLC with regards to template improvements and the projection mode.

I agree to the comment from the project maintainers that the SDL templates can be improved in different ways allowing app developers to create great applications with an awesome graphical user interface. Ideas that I would add to the list of the project maintainers are

• Enhanced text layout allowing app developers to impact the font of text fields beyond bold, italic etc.
• Template layouts allowing apps to specify the window layout using ... grids or horizontal/vertical separators where text fields, image views and buttons can be added as needed.
• Responsive layouts which can be seen as an extension to layouts as one templates can fit to many different types of screens.
• HMI theming which is different than just unify the template layout but allow apps to specify the layout and the OEMs to specify the window theme.
• Web based templates where a web view can be projected controlled by the SDL libraries using web technology allowing SDLC templates. This idea could go together with HMI theming.

Please don't forget that these are my personal ideas I have in mind that could be an awesome enhancement to the UI component of SDL. But I know all these enhancements are really difficult to realize for many different reasons. Still I would love to see enhancements to the templates become reality.

The mobile projection mode was proposed for more advanced applications. I'm not sure how far I can speak about use cases before I get into trouble but we are looking for the ability of showing complex controls like sensor meters (g-force etc.) that are going beyond text, image and buttons. This is really difficult to solve in a template approach. Other use cases are to show videos e.g. from cameras. Also customized map solutions are considered (though this is more NAVIGATION related). We are looking for the ability to implement OEM specific or non-consumer features with the new in-vehicle application platform. The web presentation allows us to move forward with delivering the demand in a realistic period of time.

I want to let everyone know that I believe that the template improvements can benefit from the web based presentation mode as a tool for any kind of application independent from where they are running (mobile, cloud, in-vehicle). Creating templates for the HMI and make them robust is a difficult task and transfer the complexity would be a gate with many opportunities for template improvements.

This vision of an SDL user interface can only be reached step by step. Starting with the projection mode in a constrained environment is a start and I believe it will be inspiring and educational for improving the design and layout of SDL templates.

9. The flow would only change minimally. If a webengine app is in HMI_FULL the HMI would not keep the app's browser view hidden but show it in the viewport on the HMI. The flow of the app's HMI level doesn't change. I can modify the flow that is included in SDL-0240 and mention where the HMI could make the web application visible on the HMI. I'll come back to you once this is done.

[theresalech]

Hi @kshala-ford - please find collective comments from the PM below. Thanks!

1.
Sounds good, thank you!

2.
With this feature it would be even more important to provide capability information to the app and properly reply to Show requests.

We already provide the display capabilities for an app in the RAI response. If the head unit does not support show requests for a webengine app, the display capabilities will let the app know. It is not necessary to say webengine apps cannot send shows, when that information can be propagated to the app by the head unit's declaration that those fields are not supported for the main window or the selected template.

3.
What is stopping a webengine app from wanting to be a traditional PROJECTION app and stream video to the head unit? Taking that question into consideration complicates this even further and suggests we need a new app hmi type.

If we keep PROJECTION as the app hmi type for webengine apps, then in Core there must be a new application API named something like webengine_projection_enabled(). This will return true if the app hmi type is PROJECTION and some secondary form of identification that it is a webengine app.

We don't believe that the lack of a secondary transport is enough of a reason for determining if a PROJECTION app is a webengine app or not. There could be non webengine apps that use the websocket transport in the future. There needs to be an additional identifier of the app type or we should use a different app hmi type.

Also the existing Core application api mobile_projection_enabled() needs to be extended to consider if the app is webengine or not. This method is called in too many cases to list here. Currently it only checks if the app hmi type is PROJECTION.

Our suggestion is to create a new app hmi type for webengine projection apps.

4/5.
Sounds good, thank you!

6.
We think this may have been missed in your initial response. Please address.

7.
So if we return to the point of contention on being able to address potential issues with other proposals and take a look at one of the latest comments from this proposal's author on SDL-0275, it seems we are in agreement that proposal issues can be debated and raised referencing other proposals currently in the process. The default behavior for the screen manager is to still assume there is a "main window", if the capabilities aren't available it has default values it uses for items like text. Should we update these default values? Should we enforce that the capabilities have to be supported and sent? Currently there is actually no hard requirement that the display capabilities have to be supported and a null value could be returned but the screen manager would still attempt to work through the lack of knowledge. Even if the capabilities were included but set all values to false/null, the screen manager would still be available but everything would fail. I'm wondering if there is a better way to handle this as it is not intuitive for a developer in my opinion. Also, there is no working implementation of including the widget windows into the screen managers which is one of the pointed out features that still works with projection. And while yes, it does "work", the amount of code that it would take for a developer to implement a robust, OEM agnostic implementation could be very large and complicated. I think the outcome to number 2 would have sufficient information to resolve this issue.

8.
My point here is that I would much rather see the SDLC focus on these improvements instead of moving forward with this type of solution. There is nothing stopping us from creating new templates that support advanced UX designs. The author has made it clear they feel the projection mode is more important. We simply do not agree on priority and as stated previously; no further discussion is necessary on the point.

9.
Waiting for the author to respond with updated flow.

10.
Given that PROJECTION and NAVIGATION currently use the NAV_FULLSCREEN_MAP template, when doing "PROJECTION" in this manner, a template type should be defined. So that, for example, an app could switch to a built in template and back.

[ghost]

With regards to 9. We are not planning to change anything of the app activation flow except that the app's WebEngine tab is shown on the HMI

[theresalech]

The Steering Committee voted to keep this proposal in review, to allow the author time to respond to the comments provided, and to provide additional time for other SDLC Members to review and provide feedback.

[ghost]

2, I think the PM is misunderstanding my point. but this item can be closed as it becomes invalid considering the solution regarding 10.

3, I need more time to look into this item regarding the ability to identify a webengine app. This proposal is not in a hurry and I would rather try to avoid adding a very webengine specific HMI type and reuse what's already available.

What is stopping a webengine app from wanting to be a traditional PROJECTION app and stream video to the head unit?

I may be wrong but I don't believe a locally running WebEngine app should capture the DOM and stream it through SDL as a video back to the HMI to be projected. I don't even believe web applications can do video streaming without the need of browser extensions.

6, I did recognize the PM comment and do understand your concern but they are not valid. I don't understand why the PM is bringing up these concerns but silently accept the original projection mode proposal.

7, I appreciate looking into my comments and providing valid feedback. I think I can speak for all members of my organization that we always support resolving issues even across proposals currently in review. I want to note that not doing so became common practice in the past and that's what I was referring to while still providing feedback that the managers should respect window capabilities independently of this proposal.

If you agree I would like to skip (and potentially invalidate) this item and invite you to have a look at 10. I think the template variant invalidates a lot of my previous expectations and your concerns.

10, You're bringing up a very good point here and I spend some time understanding how PROJECTION and NAVIGATION apps work at the moment. Please bear with me if I still misunderstand some items but I would like to stay close to the existing implementation and suggest a proposal revision to use a projection template name instead. NAV_FULLSCREEN_MAP sounds really odd for projection apps and may be a template called WEBENGINE_PROJECTION should be added.

When activating WebEngine apps with HMI type PROJECTION the HMI should initially start the app with the new template for the main window. This new template behaves similar to the video projection template but it shows the WebEngine app's DOM instead of rendering a video. The HMI can provide specific window capabilities when the main window is set to this new template. The HMI should still compute Show requests. The HMI should allow apps to switch to regular templates using Show.templateConfiguration. The templates should show the content (text, buttons, graphics) as specified by the app in previously sent Show requests. Please let me know your thoughts.

(Still 10.) think above suggestion makes item 2 and 7 invalid as the template approach matches existing window capabilities mechanism and Show RPCs are still allowed for the main window.

EDIT: (still 10 ^^) The new template should only be allowed for the default main window.

[theresalech]

A quorum was not present at the 2020-02-04 Steering Committee meeting, so a vote on this proposal could not take place. This proposal will remain in review until our next meeting on 2020-02-11.

[theresalech]

Hi @kshala-ford - please find collective comments from the PM below. Thanks!

3.
We believe the AppHMIType enum values are not supposed to apply to multiple features. All of the app hmi types that exist correspond to one specific type of application or behavior. We feel that creating a specific AppHMIType for WebEngine projection is necessary.

6.
While the author might not see this is valid, the PM has brought it as a point of contention and therefore believe it to be valid. I'm sorry, but we will have to ask you to provide feedback again. Thank you! Speaking for Joey who brought up the point, he was personally not able to be available during the review of the mentioned proposal so there was no chance to voice such concerns.

9.
Thank you for the flow chart. We still have a question about what happens to an app when it is deactivated into hmi level NONE. Should the WebEngine tab and connection to Core be closed?

10.
We agree to creating a new template layout such as WEBENGINE_PROJECTION or WEBENGINE_FULLSCREEN (or other name). We agree that the window capabilities for this template will be used to compute what contents of show are supported.

In the effort to maintain official templates across OEM platforms, we should design a high level wireframe of what components this new WebEngine projection template will include (this is not a UI design doc; it is basic requirements for implementers of the template). We think some important components to include in the wireframe are: the "return to app list" button – this should not be app defined, the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication), and app/template title.

[theresalech]

The Steering Committee voted to keep this proposal in review to allow additional discussion on the review issue. It was again requested that all SDLC Members review and provide their feedback on this proposal.

[ghost]

I'm sorry for my late reply. Due to other activities it's not easy to reply properly to every item in your comment.

The App HMI type defines app features. Not the technology (streaming video vs. rendering html). We shouldn't imply the underlying technology from the HMI type. SDL Core can easily distinguish web apps from any other app and therefore behave accordingly to the expected technology to be used. Core shouldn't expect a video and audio stream for local web applications in an SDL manner. Adding a new HMI type is not necessary and causes impact across many different SDL areas including policy. I encourage the PM to also consider implementation effort when suggesting changes. If the PM doesn't agree I would like to have the steering committee to vote. If the vote is to add an HMI type I want consensus on who will be working on the impacted platforms.

If I understand it correctly if Joey's concerns couldn't be brought up in the original mobile projection proposal he or the PM are generally against any sort of app projection (except navigation). If this is the case then please consider revisiting the original proposal and suggest to revote rejecting the proposal. WebEngine projection mode is no different to mobile projection except the technology in use is different. This is the only purpose of this very proposal to use web rendering instead of video streaming but the feature stays the same. If you bring up concerns to the feature then they also apply to mobile projection as well.

The connection to Core should stay open. This is independent of the app HMI type and also applies to default web applications.

I think we can agree on the template based approach though I don't agree to the naming. The name should be applicable to mobile projection as well and should be called PROJECTION or something else more streamlined. I would suggest to return for revision with the agreement to use the template method: When the template is used the app's web page should be presented.

[theresalech]

During the 2020-02-18 meeting, the Steering Committee voted to keep this proposal in review, to allow for further discussion on this review issue.

[ghost]

Adding 11. to the notes about what should be part of the template:

We think some important components to include in the wireframe are: the "return to app list" button – this should not be app defined,

I think this is very HMI dependent. I can say that Ford HMI is not showing such button or any other button on top of the app's viewport. However a dominant "Apps" button exist that guides the user to the app list. I think we agree here as long as the HMI designer makes sure that access to the app list is possible.

the add command menu button (or a requirement for the app to implement ShowAppMenu or otherwise implement CloseApplication),

I do agree. For 3rd party this should be a requirement but in the scope of this proposal addressing 1st party I am hesitant in accepting boundaries as I can see any business case to keep the app open like a fleet app being more important than follow these requirements. Think of car rental apps or OEM apps that want to be present permanently.

and app/template title.

This is again very HMI. But I agree to that HMI designers should consider the template title being shown outside of the app viewport, may be above the app.

[theresalech]

Hi @kshala-ford - please find collective comments from the PM regarding most comments below. We'll need more time to review and respond to 11. Thanks!

3.
The Project Maintainer always works to consider implementation effort when reviewing proposals, and the effort required to add an element to an enum set is minimal. Additionally, we believe reusing an existing AppHMIType would result in more implementation effort than creating a new AppHMIType. By the author’s definition that the AppHMIType enum defines features and not technology it should be noted that the proposed feature is not projecting a UI, it is in fact using a custom, brought in UI that the system renders. Since these features are not the same thing (playback of video vs render of UI), it is hopefully agreed that there is a need for a new AppHMIType that clearly defines two different features. Not adding a new AppHMIType also hems us in and doesn't allow new features. For example, imagine that we added a feature for mobile apps to push HTML to the head unit. Would this be a PROJECTION app? How would the system differentiate? We suggest an AppHMIType of CUSTOM_HMI_HTML

6.
To reference the original concern:

I do believe OEMs should be able to use all the features of SDL, like this proposal, as they wish when it comes to their own apps. What they do for their own apps does not affect other OEMs implementations of SDL (in most cases). What I do worry about is the effect a proposal like this will have on the ecosystem for third party apps. While there's an argument that this is operating in the same way the SDL-0031 works, I believe that is not completely accurate.
With this being for locally stored applications, there is not as much of an adverse effect to circumvent the SDLC app certification and could therefore be implemented before the SDLC agrees upon the guidelines for these types of apps. This would put any OEM who is following our path to a disadvantage compared to the OEM(s) who have chosen to ignore it. We simply do not have a way to effectively monitor and enforce this. Maybe the SDLC members do not feel this is a concern though. I can imagine Ford doesn't, but would like to hear from other members specifically.

I think the feedback provided does not address the real concern. There is no way to enforce only OEM apps to use this feature. These types of apps can be deployed directly to OEMs' head units and circumvent the SDLC entirely; this makes it possible for OEMs to create one off integrations with app partners. Therefore it is plausible that the app partners will require this type of integration (HTML rendering) for OEMs to get access to their application. This puts the ecosystem in jeopardy if it does not align with the OEMs strategy.

We would really like to get other members' feedback on this and make sure their plans align to including this feature.

9.
We can agree that the connection can stay open on HMI level NONE as long as there is a way for the HMI to be able to unregister and disconnect an app in case of performance issues (eg too many webengine apps running at one time).

To reference cloud applications, the HMI has a method to disconnect cloud apps in the event the app is using too much of the users data (ApplicationExitReason::CLOSE_CLOUD_CONNECTION). We would like to suggest adding a new HMI ApplicationExitReason and a new Mobile AppInterfaceUnregisteredReason that can be used by the hmi to disconnect webengine applications. ApplicationExitReason::RESOURCE_CONSTRAINT, AppInterfaceUnregisteredReason::RESOURCE_CONSTRAINT.

10.
As noted in (3), because the feature is not technically "projection" in that it's not projecting an existing UI or screen to the head unit, and because that terminology is already present in SDL to mean a different feature, we recommend avoiding that terminology here. We'd recommend "FULLSCREEN_HTML" or something similar, but are open to other suggestions!

[ghost]

3.
My major concern is that adding a new HMI type has impact to basically every other SDL repository for a WebEngine specific feature. Originally this proposal was suggesting a new type but it was changed for this very reason. I understand that Core complexity increases. Therefore I would like the PM to provide an estimate for the other repos. Sure thing on the SDKs it's basically just adding a new enum element but more interested into the impact to SDL Server, SHAID but also generic HMI.

The only reason why HMI type is being used is because the app's initial template should be the app's webpage. Any other solution is greatly appreciated. Possibly a new param could support the same feature and would help for other platforms. The param could be called RegisterAppInterface.templateConfiguration allowing an app to set the initial template. Not sure how to protect the web view template from apps without permissions. This change would allow to deprecate color schemes from RAI as they are part of TemplateConfiguration.

6.
Again, I think the PM needs to compare the concerns with mobile projection.

There is no way to enforce only OEM apps to use this feature. [...]; this makes it possible for OEMs to create one off integrations with app partners.

Same applies for mobile projection apps. These types of apps can be deployed on Apple AppStore and Google PlayStore and circumvent the SDLC as well. But no one is considering as it would be completely against any consortium agreement.

Therefore it is plausible that the app partners will require this type of integration (HTML rendering) for OEMs to get access to their application. This puts the ecosystem in jeopardy if it does not align with the OEMs strategy.

I'm not sure if I'm understanding. Are the PM saying that app partners would require permission to render HTML instead of using templates? Again this is not in our interest to allow 3rd party applications. Our first goal is to support OEM and commercial applications and then work on addressing template issues before we start with 3rd party. There are plenty of ways to improve templates if you look at the list I already provided. But we have to proceed on this proposal in the proposed context.

9.
Agree. Adding an exit reason is independent of the projection mode but makes sense for WebEngine apps generally. Add ApplicationExitReason::RESOURCE_CONTRAINT to HMI API and AppInterfaceUnregisteredReason::RESOURCE_CONSTRIAINT to MOBILE API.

If the IVI is 1. running out of RAM or 2. the application uses too much CPU or 3. caused excessive data traffic the application should be closed. The IVI would decide on exiting an application due to hardware constraints.

I am thinking of adding reasons per resource issue. If I'm not mistaken the exit reason should be tracked in the policy statistics. When reviewing the statistics it would be helpful to better understand the exit reason of an application. Any suggestions?

10.
my suggestion for a template name would be WEB_VIEW.

[theresalech]

The Steering Committee voted to return this proposal for the following revisions:

• (Comment 1) Add "The app HMI type PROJECTION should be enabled for in-vehicle apps. When in-vehicle apps with this HMI type are activated, the HMI should make the web page of this app visible on the screen. This web page will become the main window of the application."
• (Comment 3) Use HMI type in addition to PROJECTION, and use PM's suggestion for renaming
• (Comment 4) Add to motivation section, including: "When we planned and designed the feature we discussed this point and we believe there are plenty of SDL features that are useful. The most important items are app policy management and app lifecycle. We would like to streamline the implementation to manage activating apps, deactivating or closing apps or run apps in background. Besides the benefits for us implementing projection apps, it's also beneficial for the projection app developer to use all the SDL features like app services, remote control, vehicle data widgets and many more."
• (Comment 9) Specify that when an app is deactivated into HMI level NONE, the connection to Core can stay open as long as there is a way for the HMI to be able to unregister and disconnect an app in case of performance issues. Additionally, add ApplicationExitReason::RESOURCE_CONTRAINT to HMI API and AppInterfaceUnregisteredReason::RESOURCE_CONSTRIAINT to MOBILE API.
• (Comment 10) Use template based approach, with name WEB_VIEW.

The following items will need to be discussed when the revised proposal is back in review: Comments 6 and 11

[Jack-Byrne]

To reiterate on parts of @AByzhynar comment:

The content sent in Show request to the main window is saved internally by the HMI independently of the current active template on the main window.
So if the active template is TILES_ONLY and the app Sends Show with some text fields, the HMI saves the received text fields but did not show them.
When the app changes the active template to the other one with the support of text fields, the HMI shows text fields received beforehand.

This part is all correct to my understanding of how HMI should implement the Show rpc. This existing behavior is why WEB_VIEW should be able to receive show requests even without text fields, and be able to update previously created duplicate widgets. Even if the managers wont let this case happen, thats fine, we just wanted to preserve existing behavior for Show.

The goal to keep things simple was to try and not change widgets behavior at all for this proposal. The one exception in this proposal for widgets is that WEB_VIEW should not be able to create a duplicate widget from the main window.

[theresalech]

Hi @AByzhynar, I've summarized the status of comments below, with feedback from the PM on the open items:

Open Items
16. I think that this item still needs to be included in this proposal and not removed. This is a different proposal to the WebEngine one; this proposal is specifically about the WebView template and I fear that if this requirement is not included in the proposal then somebody implementing it will miss the requirement.

19. Given https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0273-webengine-projection-mode.md#1-template-web_view, please remove "Widgets are not affected by this proposal." from https://github.com/smartdevicelink/sdl_evolution/blob/master/proposals/0273-webengine-projection-mode.md#13-what-about-widgets.

Author to Update
17. Author to use backticks throughout proposal to mark all RPC names, e.g. AddCommand.

18. Author to update following line:

If the app has sent any AddCommands or AddSubmenus, then the MENU button should appear but must not overlay the app's content.

21. Author to remove the following:

At the time of this proposal being in review, a set of driver distraction rules are being created and proposed to enable 3rd party using the projection mode. The following bullet points are items that will be described further in the ruleset

22. Author to add bullets to Potential downsides section:

• The same downsides apply as for SDL 0031 Mobile Projection because WebEngine applications that use the web view are responsible just as any projection or mobile navigation application.

• The proposal states:
  If a WebEngine application attempts to register with this HMI type but the local policy table doesn’t allow, Core should not allow the app to register.
  This can be seen as a downside as it could break with apps being used for the first time. This feature would require the IVI to implement policy table updates through the vehicle modem instead of using a mobile application. It also requires the HMI to use SetAppProperties upon app installation to add the app policy ID to the policy table in order to trigger the policy table update through the mode.

23. In Impact on existing code section, author to change

As a new enum element is added to the AppHMIType enum, other platforms are impacted by this very minor change.
Core would need to specially treat apps with this HMI type as they are not allowed to register unless permission was granted.
Possibly SDL Server and SHAID are affected as HMI types are part of the policy system.

to

Additional enum elements have been added to the MOBILE and HMI APIs.
This means there will be a minor version change to all app library platforms, and Core.
Core would need to specially treat apps with this HMI type as they are not allowed to register unless permission was granted.
SDL Server and SHAID are affected as HMI types are part of the policy system.

Closed/No Action Required
20. We understand, and are in agreement with no further changes required for this item.

[AByzhynar]

16. I think that this item still needs to be included in this proposal and not removed. This is a different proposal to the WebEngine one; this proposal is specifically about the WebView template and I fear that if this requirement is not included in the proposal then somebody implementing it will miss the requirement.

@theresalech
16.
Could you be so kind and clarify the expected functionality from "Add command menu" button on the HMI?
Is it the same as the already described "MENU" button in the proposal here?
If yes, why do we need both of them?
Thank you

[joeljfischer]

@AByzhynar 16. It is the menu button you describe. I think that the statement should continue to include it in order to have the full list of required buttons in one place. It's fine to then expand upon the menu button in the App Deactivation section you mentioned.

[AByzhynar]

@joeljfischer Thank you. Now it is clear. I just would like to propose to name it “MENU” button (or “App Menu” ), but the same name should be used everywhere in the proposal to avoid confusion.
Don’t you mind?

[joeljfischer]

@AByzhynar I think that sounds like a good change. Maybe you should define it in parenthesis the first time it’s used? It should be obvious, I just want to avoid confusion between that button and one the developer makes themselves using the WEB_VIEW.

Something like

“MENU (the button that shows the list of AddCommand and AddSubmenu items)”

[theresalech]

The Steering Committee voted to keep this proposal in review, to allow the PM and author to resolve the remaining open items in discussion (16 and 19).

[ghost]

@AByzhynar and I came up with a conclusion to 16 and 19. Both are very closely related to 15 and 18 which is why we decided to use a title instead. We're OK with new numbers (24, 25) if further discussion is needed but we try to provide a complete statement to both (actually all four) items:

---

16. & 18. App Menu

We want to make everyone aware of the accepted proposal SDL-0116 Open Menu which added a new RPC called ShowAppMenu (see proposal) This proposal states the following:

The proposed solution is to specify the built-in "menu" button to not exist anymore at all for projection apps. Instead those apps should be allowed to send a request to show the built-in menu view.

and

Projection apps should project a button which the user can press. The app would react on this button press and send the ShowAppMenu request.

Apps which show content using WEB_VIEW have the same responsibilities than projection apps therefore the same requirement applies to WebEngine apps. We suggest to add the following items to the proposal in favor of showing a menu button on the HMI:

1. The HMI should include an app title and a template title. The template/app title should be visible when the app is activated. The exact location of the template/app title relies on the OEM.
2. The HMI should include a button to access the app list
3. The HMI should not show a menu button on the HMI. Instead the app should be required to behave as followed:
   1. If the app uses AddCommand or AddSubMenu the app is required to provide an own menu button on the WEB_VIEW and send ShowAppMenu if this menu button is selected by the user.
   2. If the app does not use commands or sub menus the app is required to provide a way to the user to exit the application. The app should use CloseApplication RPC for this.

---

15. & 19. Related to widgets and duplicate content

With regards to widgets we think that the statement that's currently in the proposal DuplicateUpdatesFromWindowID=0 is not allowed for widgets. is weak. As already commented the developer can change to a template layout, add the duplicating widget and turn back to WEB_VIEW very easily. Therefore we suggest to remove this statement from the proposal.

The section What about widgets? is correct and accurate to the current behavior of Core. The behavior of the libraries is to be seen separately and also affects mobile navigation/projection apps and should be revisited separately from this proposal. This aligns with Jack's comment here.

Our suggestion is not to change the behavior of how SDL Core is duplicating content. The section What about widgets? already makes this suggestion.

[AByzhynar]

21. Author to remove the following:

At the time of this proposal being in review, a set of driver distraction rules are being created and proposed to enable 3rd party using the projection mode.
The following bullet points are items that will be described further in the ruleset

@theresalech The enabling of 3rd party apps is a long term goal which we hope to enable
with a future proposal.
We would like to keep it in the proposal, but move to the motivation section

At the time of this proposal being in review, 
a set of driver distraction rules are being created and proposed to have 
the possibility to enable 3rd party using the projection mode in the future. 
The following bullet points are items that will be described further in the ruleset:

- minimum font size
- minimum contrast between font and background
- min/max brightness (for day and night mode)
- number of buttons
- minimum size of a button
- no customized keyboard
- no video playback (exceptions in standstill per countries)
- NHTSA related guidelines
    - Amount of text (button and text fields)
    - number of lines of text
    - Complexity of use cases (number of steps to fulfill a use case)

More items may be included in the ruleset as they become Driver Distraction affected.

[AByzhynar]

@theresalech
Let's assign a new number : 26 to the current item.

We consider that

The WEB_VIEW template is not allowed to to support textfields, softbuttons, graphics (inside or outside of the webview).

is not accurate enough and should be changed to :

The WEB_VIEW template must be designed without support showing the content of the Show RPC like textfields, softbuttons, graphics (inside or outside of the webview).

[AByzhynar]

@theresalech The changes proposed in 16. & 18 here: #912 (comment) will also require to remove application deactivation section

[theresalech]

Hi @AByzhynar and @kshala-ford, thanks for your feedback. We are okay with your suggestions for 24 and 25. I've included an updated status of comments below, with feedback from the PM on the open items:

Open Items
21. This proposal is specific to OEM applications, and therefore this motivation is out of scope for the proposal and should be removed.

26. We'd recommend

The WEB_VIEW template must not declare support in its WindowCapabilities for any softButtonCapabilities, the TextFields with name: mainField1-4, mediaClock, mediaTrack, nor the ImageFields with name: softButtonImage, menuIcon, graphic, or secondaryGraphic.

Author to Update
17. Author to use backticks throughout proposal to mark all RPC names, e.g. AddCommand.

22. Author to add bullets to Potential downsides section:

• The same downsides apply as for SDL 0031 Mobile Projection because WebEngine applications that use the web view are responsible just as any projection or mobile navigation application.

• The proposal states:
  If a WebEngine application attempts to register with this HMI type but the local policy table doesn’t allow, Core should not allow the app to register.
  This can be seen as a downside as it could break with apps being used for the first time. This feature would require the IVI to implement policy table updates through the vehicle modem instead of using a mobile application. It also requires the HMI to use SetAppProperties upon app installation to add the app policy ID to the policy table in order to trigger the policy table update through the mode.

23. In Impact on existing code section, author to change

As a new enum element is added to the AppHMIType enum, other platforms are impacted by this very minor change.
Core would need to specially treat apps with this HMI type as they are not allowed to register unless permission was granted.
Possibly SDL Server and SHAID are affected as HMI types are part of the policy system.

to

Additional enum elements have been added to the MOBILE and HMI APIs.
This means there will be a minor version change to all app library platforms, and Core.
Core would need to specially treat apps with this HMI type as they are not allowed to register unless permission was granted.
SDL Server and SHAID are affected as HMI types are part of the policy system.

24 (formerly 16 and 18). Author to add the following items to the proposal in favor of showing a menu button on the HMI:

1. The HMI should include an app title and a template title. The template/app title should be visible when the app is activated. The exact location of the template/app title relies on the OEM.
2. The HMI should include a button to access the app list
3. The HMI should not show a menu button on the HMI. Instead the app should be required to behave as followed:
   1. If the app uses AddCommand or AddSubMenu the app is required to provide an own menu button on the WEB_VIEW and send ShowAppMenu if this menu button is selected by the user.
   2. If the app does not use commands or sub menus the app is required to provide a way to the user to exit the application. The app should use CloseApplication RPC for this.

The application deactivation section will also be removed.

25 (formerly 15 and 19). Author to remove DuplicateUpdatesFromWindowID=0 is not allowed for widgets. Behavior of how Core is duplicating content will not change, so the What about widgets? section can remain as is.

Closed/No Action Required
20. We understand, and are in agreement with no further changes required for this item.

[AByzhynar]

21. This proposal is specific to OEM applications, and therefore this motivation is out of scope for the proposal and should be removed.

We agree to remove it but our long term goal is still valid for the future proposals.

[theresalech]

Hi @AByzhynar, can you please advise if you are okay with our suggestion for Item 26?

[AByzhynar]

@theresalech All the changes proposed by you in #912 (comment), including item 26, are included in the new revision.
PR with the new revision: #1023
So we are okay with all the proposed changes. Thank you.

[theresalech]

In preparation for today's Steering Committee meeting, I have summarized the revisions the PM and author have agreed upon below. Unless there are any other comments from members, we'll ask the Steering Committee to vote to accept the proposal with the revisions below:

17. Author to use backticks throughout proposal to mark all RPC names, e.g. AddCommand.

21. Author to remove the following

At the time of this proposal being in review, a set of driver distraction rules are being created and proposed to enable 3rd party using the projection mode.
The following bullet points are items that will be described further in the ruleset

22. Author to add bullets to Potential downsides section:

• The same downsides apply as for SDL 0031 Mobile Projection because WebEngine applications that use the web view are responsible just as any projection or mobile navigation application.

• The proposal states:
  If a WebEngine application attempts to register with this HMI type but the local policy table doesn’t allow, Core should not allow the app to register.
  This can be seen as a downside as it could break with apps being used for the first time. This feature would require the IVI to implement policy table updates through the vehicle modem instead of using a mobile application. It also requires the HMI to use SetAppProperties upon app installation to add the app policy ID to the policy table in order to trigger the policy table update through the mode.

23. In Impact on existing code section, author to change

As a new enum element is added to the AppHMIType enum, other platforms are impacted by this very minor change.
Core would need to specially treat apps with this HMI type as they are not allowed to register unless permission was granted.
Possibly SDL Server and SHAID are affected as HMI types are part of the policy system.

to

Additional enum elements have been added to the MOBILE and HMI APIs.
This means there will be a minor version change to all app library platforms, and Core.
Core would need to specially treat apps with this HMI type as they are not allowed to register unless permission was granted.
SDL Server and SHAID are affected as HMI types are part of the policy system.

24 (formerly 16 and 18). Author to add the following items to the proposal in favor of showing a menu button on the HMI:

1. The HMI should include an app title and a template title. The template/app title should be visible when the app is activated. The exact location of the template/app title relies on the OEM.
2. The HMI should include a button to access the app list
3. The HMI should not show a menu button on the HMI. Instead the app should be required to behave as followed:
   1. If the app uses AddCommand or AddSubMenu the app is required to provide an own menu button on the WEB_VIEW and send ShowAppMenu if this menu button is selected by the user.
   2. If the app does not use commands or sub menus the app is required to provide a way to the user to exit the application. The app should use CloseApplication RPC for this.

The application deactivation section will also be removed.

25 (formerly 15 and 19). Author to remove DuplicateUpdatesFromWindowID=0 is not allowed for widgets. Behavior of how Core is duplicating content will not change, so the What about widgets? section can remain as is.

26. Author to update

The WEB_VIEW template is not allowed to to support textfields, softbuttons, graphics (inside or outside of the webview).

to

The WEB_VIEW template must not declare support in its WindowCapabilities for any softButtonCapabilities, the TextFields with name: mainField1-4, mediaClock, mediaTrack, nor the ImageFields with name: softButtonImage, menuIcon, graphic, or secondaryGraphic.

[theresalech]

The Steering Committee voted to accept this proposal with the revisions outlined in this comment.

[theresalech]

Author has updated proposal to reflect agreed upon revisions, and implementation issues have been entered:

• RPC
• Core
• SDL HMI
• Generic HMI
• iOS
• Java Suite
• JavaScript Suite
• Server