Sending an effect without transitions bricks the backlight #24
Comments
|
Adding here the report from |
|
I tried your config and the exact thing happened.
Fuck. This is pretty serious. I'm going to pull away the new effects features until we can figure out wtf is happening. |
|
My power button light still works normally, though. I'm going to try running the SteelSeries software from a windows partition and see if it can still pick the keyboard up. |
#24 There is a critical issue that LOCKS the RGB controller in an unusable state, which persists after reboot. At least 2 machines are affected, including mine. Still trying to figure out a solution to reset the controller and make it usable again, bit in the meantime I'm reverting all the new features that may have caused this.
|
Can't fix it from Windows either. SteelSeriesEngine does pick up the keyboard and acts as if everything is normal, but the lights stay off. @TauAkiou any idea ? Just to be clear, I'm not blaming you (or anyone) for this. But you probably know the HID protocol of SteelSeries better than me at this point. Do you think there's any command that could be sent to the RGB controller to get it out of this broken state ? |
|
I'm going to have to look into this. It looks like the USB device really doesn't like receiving a block that has no transitions. That's bad design on MSI's part if it breaks the entire keyboard. I'm going to look into the code that the program generates using that and take a look to see if there's a way to hard reset the controller. |
|
One thing I would try doing in the meantime is resetting the Embedded Controller, which may reset the keyboard and fix the issue. MSI provides instructions here: https://forum-en.msi.com/index.php?topic=112416.0 |
|
I already tried resetting the EC by holding the button under the laptop for up to 30 secondes (with power disconnected), with no effect. I'll try again just in case i did it wrong. EDIT: I tried again, this time letting the laptop rest for a few minutes. It did not change anything. |
|
Oh wow. This is terrible! Can I suggest to completely remove tag Is it possible you get help from official MSI people? |
|
It might be worth reaching out to MSI to secure a solution to this. There has to be a way to factory-reset the lighting controller, and if there isn't, Steelseries really messed up when it came to the software design of this keyboard. I believe the issue was caused because the above configuration contains no transitions, and for whatever reason the Steelseries device completely locks whenever it receives an effect without one. This is something I missed during testing, as I never loaded a configuration file that had no transitions attached to it. The fix is simple; make sure that an effect block has at least one transition in it before sending it to the controller. That also being said, the code broke two keyboards, and I'm still quite embarrassed that I didn't catch that during my own testing. |
|
I do not know anyone from MSI, and i'm rather skeptical about them helping us. Plus, isn't SteelSeries responsible for the fimware and protocol ? Worst case, my laptop is still under warranty, but i'd rather spend one month trying to make the keyboard work than trying to explain to my reseller (mainstream IT reseller) what happens and why they should exchange it and not reinstall/format Windows & SteelSeriesEngine (as they will probably suggest even if I don't have any Windows). But this doesn't solve @Askannz issue, and, in fact, doesn't solve the issue at all. |
|
I anyone opens a ticket on MSI website, please add the link to the ticket to this issue so we can track it |
|
I did a cursory dump of both packets to check for alignment or other issues; it seems as though my theory is correct: An effect with no transitions will break the controller. To test this, I took the broken configuration and ran it through a slightly modified version of the utility that dumped the packet to a file rather then sending it to the keyboard. There were no significant alignment differences and everything was in the right place. The only difference is that the effect blocks and millisecond fields were entirely empty. I'm going to push a change to my fork that prevents users from pushing transition-less effects to the keyboard, and add a big warning to the documentation to never push a transition-less effect lest this happen. Our next step should be seeing how we can get @rodolpheh and @Askannz 's keyboards working again. |
|
This is for a standard Steelseries Mouse/Keyboard, but these instructions might help, since they are in a sense the same kind of device: https://linustechtips.com/main/topic/172028-how-to-fix-steelseries-firmware/ |
|
I don't have a Windows partition but I will try to mess around with a VM. I'll let you know if I notice any changes. |
|
FWIW I never could get SteelSeries to detect the keyboard from within a VM. I tried messing around with the device manager in Windows, but no amount of uninstalling the device made SteelSeries prompt a firmware update. I also tried going into the hidden MSI BIOS but no "magic reset button" jumped out. There's stuff related to USB but I don't understand half of it. |
|
Uh, whatever I did on the Windows side seems to have made things worse on Linux: https://pastebin.com/5QmHH6UM |
|
Fucking yikes. This is a pretty serious firmware issue on Steelseries' side if a malformed command is completely hosing the controller like this. I have a trouble ticket with Steelseries open over how to perform a hard reset or how to reupload the firmware. If there's any information you can suggest I bring up, post it here. I'm still debating over whether or not to link them to this project/issue. I'm also debating dropping all of the effects code and simply leaving the documentation. If a malformed command to the effects engine can break the interface with no chance at recovery, then there are likely to be more issues that can cause similar problems. |
|
The USB errors wouldn't go away even after a reboot, but I left the laptop off overnight and now it's back to the state where it's silently accepting HID commands. Weird. Makes me think there may be a timed overcurrent protection tripping off somewhere.
Thanks, hopefully they'll respond. I mean, if we can hose the controller from Linux like this, then a borked update to their own software could potentially do it too, so maybe they'll take this seriously. |
|
This is a shot in the dark, but if the device seems like it's accepting commands, at the least, would it be possible to send it a set of fresh effects and reset the internal tables that way? The effects code assigns effect numbers sequentially; if we send a packet to assign all keys to a new, working effect or force-overwrite effect slot 0 with a fresh E0, would that be enough to force the keyboard back into a operable state? An effects file like: |
|
That's a good idea, but it didn't work sadly. I tried your config but the keyboard just ate it up and nothing changed. Same thing when adding more effect slots. As a last resort I even tried the original config that caused this mess (can't make it worse right ?), but that didn't do anything either, not even an error like last time. |
Are you referring to the commands that Steelseries sends when starting the app? I noticed that it those packets it seems to reset the effects packet, but if that was the case, then why doesn't the keyboard lock up like @Askannz and @rodolpheh? |
|
@Askannz and @rodolpheh ran the configuration file listed at the top of the list that had no transitions attached to it. This, for whatever reason, appears to be interfering with any commands being sent. My theory is that the controller firmware doesn't properly handle 0ms length effects or 0ms transitions properly. A zero-ms long effect or an effect with no transition blocks (this might also be a problem with 0ms transitions as well) could be causing a logic error in the firmware that crashes the controller. Since MSI/Steeleseries made the smart decision to store the previously loaded effect/key blocks in NVRAM, the controller will reload the bad NVRAM block and lock up the controller again. |
|
Dang, well I guess I am still confused as to why the SteelSeries Engine sends effects package with no transitions or anything, except for the 0b command at the beginning and the effect id. Do you think SteelSeries is resetting the effects corresponding to the ID? Edit: Wait, have they tried just setting all the keys as Steady and reset all the effects package? Or like they literally cannot send any keys? |
MSI GE63VR-7RE
I used Wireshark on Windows to dump the binary packets sent by SteelSeriesEngine. If you're referring to https://github.com/Askannz/msi-perkeyrgb/tree/master/documentation/utils, those are just convenience scripts for visualizing the dumped packets, not capturing them. |
|
I'm still communicating with SteelSeries over this; no updates thus far, but I hope we can find a solution and fix the broken keyboards. I'm going to continue work on the Effects engine, but I'm keeping it in my fork until we can fix these keyboards and make sure it doesn't happen again. I'm still holding hope that we can find a solution to this. |
|
What we need here is a way of clearing the keyboard's NVRAM, not the firmware. You would think that they would offer a way to do that as a recovery or reset mechanism, but apparently not. @ErrorErrorError I added a check in my branch (the original effects branch) to fail if there isn't at least one transition: TauAkiou@7d32728 More checking might be necessary to make absolutely sure this state is never passed to the keyboard, but that isn't going to recover the keyboards that were bricked by this. |
It couldn't be an issue with EC also? Because you can still communicate with the keyboard but it does not seem to write the bits to the NVRAM or overwriting it. I did notice that sometimes SteelSeries sends a different initial package number when writing ouput to HID. msi-perkeyrgb/msi_perkeyrgb/msiprotocol.py Line 170 in 87c92ac On my system it sends |
|
MSI says they can't support that, SteelSeries says to contact MSI because they don't do anything, only the SteelSeries Engine software (which according to a recent article on Gizmodo, is a piece of junk with security holes). I think it's not worth trying to get help from either of them. |
|
Hello. It is not direct for you keyboards, but may be give you some insigths https://www.youtube.com/watch?v=iMP9mGKfQw0 |
|
Have you tried this? https://us.msi.com/support/technical_details/NB_KB_SteelSeries |
It's a hardware issue, not a software issue unfortunately. |
|
Hello! I tried this software in ubuntu 22.04 and got some python errors, unfortunately I tried again. Lights flashed once and then the keyboard disappeared from lsusb Lights were not working even while booting. After a poweroff they work while booting, though not very well... but in Windows they don't work. Neither in Ubuntu, I guess I bricked my keyboard. In normal mode, the number 4 or so flashes white, the x flashed red, but nothing more. In rainbow, more keys seems to work |
|
Well after I entered the SteelSeries app and odified soe things and saved, the keyboard looks like working again except the m which was working as my previous post works. Will try messing again with that software and saving |
|
OK after setting the DEFAULT mode, clicking the M to change it, then pressing SAVE, now my keyboard is back thank god, at least in Windows and the m works.
It does seem like something is still messed up as when I boot the only one or two keys blink for a second then turn off, and in Ubuntu after logging in the power button switches from red to white for some seconds. But in Windows the backlight is working at least. I removed the udev rules in Ubuntu and removed the msi-perkeyrgb just in case it was trying to apply rules on boot but that issue is still happening, and iirc it was working correctly from boot to stop before (for Windows). But I can live with that as long as the rgb is working at least in Windows. |
|
Ello Do you know if this is an issue that could be patched so full animations are available again? |
I’m coming late here, but just to say that it’s not just SteelSeries software that’s a piece of junk, MSI software too. Moreover, their support is absolute zero when it comes to tell how their own hardware works. I wanted to know which commands are sent to the EC to get some functionality such as enabling/disabling display overdrive so I can program it myself on Linux, but since I said the magical word “Linux”, they just answered they don’t support Linux… except I didn’t ask for any help on Linux, only how the hardware works so I can implement functionality on Linux, but they don’t seem to get it or don’t want to understand. In the end I managed to enable/disable overdrive on Linux, but it’s probably not the best way to do it, thanks MSI ! It was the first time I bought from them, but as things go, with their junk programs, Windows-centric mind, lack of cooperation and some other hardware problems I encountered, I don’t see how my next laptop could be an MSI, that’s for sure. |
|
This is really necromancing here ... but ... Having read through the posts to get here; I haven't seen anyone doing a correct Electronic Controller reset. That link to MSI says 'hold the power down for 30 seconds while disconnected from mains". That isn't sufficient, and unnecessary on some models. My GE76 takes a 30 second press on the EC reset button on the machine underside while disconnected. Then on re-connecting to mains; holding the power on for five seconds or so. That has reset my EC several times. It could be some peculiarity with the GE76 since none of the other SS effect writers around github and elsewhere seem to work either; none of them even mentioning support for this model. |
|
@brokencog As I mentioned in a previous post, the backlight isn’t managed by the Embedded Controller, at least not on my GP76, otherwise the problem would be solved easily, it’s on the keyboard electronics directly. Some non-laptop keyboards have a reset procedure, but I have seen none for laptop keyboards (which doesn’t mean there is none, just that it’s not been disclosed). |
|
@NovHak-Linux I read that comment - it and several others mention 'reset' of the EC, but nothing in the descriptions would indicate the EC actually being reset ... removing the battery, flashing BIOS and holding the power button will not reset the EC. |
|
@brokencog I suppose your argument here is that the kb backlight may be managed by the EC after all, just that none of us did reset it properly, but since you did it, I guess you were able to see if it indeed had any effect on the SS keyboard configuration… so did it ? Additionally, could you point us to the source were you got the proper reset procedure ? But anyway, now that I rethink of it, it’s very unlikely that EC is in charge of this : is has too many important things in charge, such as fan management, CPU reset, battery management (in the case of my GP76 at least), to have fancy lighting added to the list. Considering SS desktop keyboards manage this internally, it’s probably the same with the laptop ones. |
|
@NovHak-Linux I am not sure if this is the same mention of it ... but this gives a summary. [It's ironic they link to the same MSI instructions which were described in this thread ...] I wasn't suggesting you were wrong (despite your snarky assumption) because a) your laptop model could very well have different circuitry and b) what required a reset on my keyboard controller might not do so on yours. However, the steps mentioned in this thread of comments does NOT mention the procedure linked above which is exactly what I wrote initially. I needed to do a reset after crashing the controller with a different program (MSIKLM) and then resetting it via the above command. To quickly determine if that method is viable for you - flip the laptop over and look for a tiny reset hole adjacent to a small, circulary-looking icon. Mine is lower left of the center screw. If so, chances are 99% likely that is the EC reset pin. Plenty of images online. Note: I have only succeed with the reset when holding the pin for 30 seconds. So, when you test it, count all the elephants ;)> |
|
We can quickly close this debate: I've done this method, this was the first one I did, because it's the most quoted one when googling how to reset EC on MSI. |
|
@rodolpheh Why is it a debate?? What's being debated?? How should I know "this was the first one you did"? Nothing in this thread described the reset procedure I linked -- and as I already wrote I don't claim it works universally, but I do claim what others have described in this thread is insufficient; I'm holding at least one hardware model for which it doesn't work. Hence my posting an additional reset mechanism. I will point out that "the most quoted one" from google responses is entirely subjective. For instance it isn't until the fifth or sixth result of searching "msi laptop ec electronic controller reset" in which I see a YouTube description of the method using the underside reset button, all the others reference the power button method. |
|
Maybe "debate" wasn't the right term, you'd have to forgive me as English isn't my native language. I appreciate that you are trying to help on a thread that is 4 years old. Just to reassure you, I tried your method right now, which I'm pretty sure I tried years ago, and it didn't fix it. I know that the EC reset method was successful as before because the computer goes into a different booting cycle (which according to some info I read years ago means that it was successfully done). You'd have to forgive me but I spent countless hours reading MSI documentations and user manual so in my memory it felt like the information was pretty out there. Maybe it came about at the tenth page of Google searches, at a time when Google didn't suck that much. |
|
Wow, I'm amazed this thread is still alive. Please keep the discussion civil. I had tried the reset button too at the time, without success. After reading @brokencog comment I just did it one more time (more out of curiosity than to fix a laptop that's quite literally falling apart at this point). The machine does seem to go into a different boot mode but the keyboard lights won't come up. |
|
@Askannz just to be clear, the reset button isn't going to change how msi-perkeyrgb functions, if it didn't change LED colors before still won't do so after the rest. Also, a confirmation that the resetting the EC worked is with the laptop power disconnected, pressing the power-on button does not do anything. One needs to re-connect the power then press the power button for a few moments. This is because battery charge level's and other stuff are also reset and the laptop won't attempt to recharge without being connected to mains. |
|
Yes I can confirm that I had to power the laptop to turn it on and upon boot it was showing a low level of battery which quickly got up. And I remember clearly the same thing happening 4 years ago when this issue was first opened. I think Askannz is aware that this wouldn't change things in msi-perkeyrgb as they are the author of msi-perkeyrgb. In case you missed it, the subject of the issue is that a wrong command bricked the backlight of our keyboards, which means that they do not have any backlight at all. Seeing it light up would indicate a massive progress, that's why Askannz was pointing out the absence of light. |
|
But if you boot into Windows and use the official MSI keyboard software to configure the keyboard, doesn't it come back to life? It happened to me and it came back with that. Except the "m" that wasn't working but worked after a remap |
|
I do not have Windows installed on this machine. In your issue however you report that the keys were lighting up on boot, which sounds like they are not completely bricked. In our case, no lights comes on at boot, and in my case even the power button uses the wrong LED. Askannz previously tried the SteelSeries software and it didn't solve their issue, might be worth trying again in case they updated their software to unbrick bricked keyboards but I'd doubt that. |
|
I read his comment about your keyboard being bricked. That's exactly why I commented. Let me make a very linear explanation of why I made the initial comment, because I don't feel the hostility (albeit somewhat mild and primarily from a single user) is warranted despite my having raised a four? year old thread without a definitive solution.
It's unfortunate it doesn't work for others -- as I wrote I never made any assertion it was a universal solution; only that this thread didn't have any indication of this "alternative" method being attempted and it seemed to me worth mentioning. Thanks for the response, obviously it's possible to control the LEDs so hopefully a solution will be found eventually. Also, @rodolpheh are you still unable to control the keyboard LEDs via the Win10 SS Engine software? |
|
My memory is not very good and I haven't gone through the whole thread, but I do not remember reading about a method that only involves pressing the power button. As a matter of fact, the first comment in which I reference the EC reset talks about holding the button under the laptop, which is the EC reset one (with power disconnected) which is what MSI recommended at the time and still recommend when searching how to reset the EC on Google. As for Windows, I unfortunately do not have Windows on this machine and do not have enough storage to dual-boot on it. |
|
my guess has always been that the MSI keyboard IC controller is a seperate USB device that powers up on system boot and operates independently from the rest of the system, not unlike any other USB device out there. unless MSI/Steelseries changed that, then I don't see any reason it wouldn't be different on later models. perhaps it might work for you, but other people that faced this bug have not had much success with it. i would not use the old version here, but instead the original fork in which I added some checking to prevent the user from entering an invalid transition combination, at your own risk: https://github.com/TauAkiou/msi-perkeyrgb i couldn't vouch whether or not this works on newer laptops, either since this was created and debugged on a MSI GS65 back in 2019. my GS65 has since broken, so it's unlikely I will be doing any more work on that branch. |
|
@rodolpheh I feel like we are describing something different?? There are two methods of resetting the EC:
Reading your recent comment, forgive me for being confused but, it doesn't seem like method 2 has been tried. I know you mentioned earlier doing so, but that comment was not clear to me either. I haven't tried using the SS Engine in a VM. I'll install Win10 VM and check the Engine works virtually. It didn't work in Wine, unsurprisingly, but I suspect a VM would succeed. @TauAkiou I agree that the EC is connected as a USB device. Otherwise we wouldn't be able to capture the interaction via Wireshark's USB interface capture. I don't know how much MSI changes internal products from model to model. I am on a GE76 from 2021 (not sure the year). |
|
I know it will make me sound crazy but Google still has in cache the information that was on MSI's website, see attached picture. Sorry for the confusion. My original comment from above did refer to the button under the laptop. EDIT: the link featured in the picture is not the right one. I've found out that MSI's website has been changed. The oldest backup from the wayback machine refers to the EC reset hole, hence the confusion. |
that's not entirely what i meant - i theorized that the keyboard itself had a NVRAM chip that stores the keyboard settings. this is how i suspect that the keyboard knows the previous effect when the laptop is powered down. i don't think this chip is connected to the EC controller in any meaningful way, which means that if the keyboard gets an invalid configuration and that configuration gets saved to NVRAM, it will attempt to load the invalid state on every boot. this is what I believed was happening with the zero transition bug. i don't know if MSI/SS fixed this issue in later hardware iterations, but it reeks of bad design. this scenario could also theoretically be caused by a bug in the Windows steelseries software as well or some other sort of corruption to a similar if not the same effect, or simply if the NVRAM ends up being somehow corrupted. |
After trying to create and use an effect through a configuration file, I completely lost the ability to have keyboard backlight, following an hidapi error :
At first reboot, the SteelSeries keyboard wasn't showing up in
lsusb. It started showing up again after a full shutdown. I tried to update the UEFI BIOS firmware (following the MSI instructions) to see if it would reset the keyboard but it did not.Here is the configuration file that made my keyboard crash and triggered the hidapi error (sorry I did not logged it and I can't make it appear again) :
My laptop model is a MSI GE65 Raider 9SE
My system is
5.3.7-arch1-1-ARCH x86_64 GNU/LinuxMy
msi-perkeyrgbversion is1.4_effects_alpha, built from the AUR this morning (which shows a bump to version 2 normally)The SteelSeries keyboard is showing in
lsusbasBus 001 Device 004: ID 1038:1122 SteelSeries ApS SteelSeries KLCAnd finally,
greping SteelSeries indmesggive me :When I now use
msi-perkeyrgb, it doesn't output anything.Did you happen to stumble on such issue ? Does anyone knows a way to solve this ?
Let me know if you need more information
The text was updated successfully, but these errors were encountered: