Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

945 refactor motor drivers #951

Merged
merged 45 commits into from
Jan 4, 2022
Merged

945 refactor motor drivers #951

merged 45 commits into from
Jan 4, 2022

Conversation

Ezward
Copy link
Contributor

@Ezward Ezward commented Nov 3, 2021

Addresses Issue #945
Companion documentation PR

This should be QA'd in conjunction with the documentation PR using the test plan in Issue 945

NOTE: I expect and desire a lot of input on this pull request. If you like it and want to approve it, then great. But please don't merge it. I'd like to get consensus on this prior to merge.

This is a large refactor of the motor drivers. The goal is to make the motor drivers more reusable. I got the itch to do this because I have a Duckiebot differential drive robot. In another branch I'm adding odometry/kinematics and pose estimation, but I could not test it on this bot; Duckiebot uses an L298N 3pin kind of motor setup and we have a drive configuration, DC_TWO_WHEEL_L298N, that almost works. The issue is that this bot uses a PCA9685 to generate the duty cycle. Actually it's worse than that, it mixes some TTL outputs from the GPIO and the PCA9685. So our driver would not work, since it assumes only GPIO output (using RPi.GPIO library) for both TTL and PWM. So I was in the position of writing yet another one-off driver. I realized that if the differential drive code took a higher-level pin specification, then it would not be tied to any particular hardware or library. Looking at actuator.py, there are a number of instances where we have almost the same driver logic, but re-implemented for different libraries or pwm hardware. Worse, they diverge a little in how they actually work. This then causes a proliferation of configuration options in myconfig.py. There is very little documentation how the hardware maps to a particular drivetrain so how does a user know which drive train to choose for their hardware? There are a number of classes for particular microcontrollers or pwm generators that are not actually hooked up or documented; they are just latent in the code; no one would know they are there or how to use them looking at the templates or myconfig.py.

So I did not want to add to this problem. What I tried to accomplish was to refactor so we only need a few reusable drivetrains; one for servo/esc, One for a server plus L298N, one for differential drive L298N 3pin wiring and one for differential L298N 2pin wiring. These drivers take pin-specifiers rather than raw pin numbers. Pin specifiers indicate the source for the pin; PCA9685, RPi.GPIO/Jetson.GPIO or PIGPIO and other particulars. These can all be used on raspberry pi or jetson nano, with our without a PCA9685. This allows us to drastically cut down on the code in actuators.py and complete.py and the amount of configuration in config.py.

So this is pretty opinionated pull request. Actuator.py has a lot of cruft in it. I would like to see it paired down to what we really intent to support in the framework. For those other edge cases, like a particular microcontroller implementation, I think those should be in their own repository with a readme that documents what they are and how the donkey code would need to be changes to use them. Maybe that repo in is in autorope, maybe it should be in the author's own github account if it is particular to them. If there is another way to handles these things, please let me know.

This pull request is the first of several I have lined up, so I hope to get a lot of input on this. That will help guide me in how I manage the subsequent pull requests. I would enjoy a lot of input. I this this also needs a lot of testing, so I have when you are reviewing this that you will pull the code and try it on your donkeycar.

Here is the approach I took:

  1. create a layer of abstraction over input/output/pwm pins. I call this the pin provider api and it can be found in donkeycar/parts/pins.py. There are 3 pin providers currently implemented; PCA9685, Rpi.GPIO (and so Jetsion.GPIO) and PIGPIO. I'm trying to figure out of we need to support ServoBlaster and Arduino Firmata (or possibly our own Arduino sketch). Pins are specified using a pin-spec string. For instance, "PCA9685.1:40.13" specifies channel 13 on the PCA9685 at address 0x40 on I2C bus 1. "RPI_GPIO.BOARD.33" specified board pin 33 driven by the RPi.GPIO library. "PIGPIO.BCM.20" specifies broadcom gpio-20 driven by the pigpio library.

  2. Refactor pwm controllers (those support set_pulse() methods) to use this new system, so we can eliminate the other library specific implementations that are tied to specific configurations. This allows us to be more general in our support of these underlying hardware architectures without users having to write their own parts. For instance, the PIGPIO_PWM configuration supports an hardware layer that takes pwm pulses for both steering and throttle as you would do for an RC car with a servo and esc. However, it requires the use of the PIGPIO libary, which does not work on Jetson Nano. There is now a drivetrain named PWM_STEERING_THROTTLE and a generalized pulse controller called PulseController that uses the pins.py pin provider api to send servo style PWM pulses to Servo/ESC style RC car drivetrains. This means we can retire PIGPIO_PWM and it's code (and I2C_SERVO drive train configuration). A car with a servo/esc drivetrain can now be driven by RPi.GPIO, PIGPIO or a PCA9685 using just the PWM_STEERING_THROTTLE drivetrain configuration.

  3. Refactor non-pwm-controller configurations, like the DC_TWO_WHEEL and DC_TWO_WHEEL_L298N, to use pin providers. This has those same benefits; now a differential drive robot can use the GPIO header or a PCA9685 for PWM. In fact, it can mix the two; I have a Duckiebot that does just that; now it can run donkeycar. Duckies and Donkeys living together in peace!

  4. Delete unused code. We have a bunch of code that is not hooked up to anything OR is supporting boards that we really don't support. So we have a bunch of code that is not in a template and has no configuration. The only way to use it is to edit your manage.py (or create a fork and edit complete.py). So that makes actuators a dumping ground for individuals' pet projects that are not really part of the framework. That makes actuators much harder to maintain. For the most part, I have not actually deleted this, but I have marked it as @deprecated and added a comment that describes why I think it should be deprecated and how we might actually rescue this code and make it part of the framework. So with the exception of the DC_TWO_WHEEL** drivetrains, the old drivetrain code remains in place, but is marked as deprecated. This allows us to merge this code into the 4.x branch without breaking changes (except to the DC_TWO_WHEEL drivetrains, as mentioned. If we think these should also not be touched, I can put the original code back in and create a parallel set of configurations for the new pin-provider versions). Come version 5.x, I believe we should either delete the deprecated code or make an effort to make it part of the framework.

  5. Add a lot more commenting and documentation while we are at it. I've added a lot of comments to the code. There will be a second pull request for donkeydocs after this pull request has gone through some review. Again, given that the old configurations are generally maintained, the old docs are still valid.

Here is a high level explanation of the file changes:

  • donkeycar/parts/pins.py : this is a new file that contains that Pin Provider api and it's implementations.
  • donkeycar/utilities/deprecated.py : adds a decorator for classes and functions that are deprecated. This will log a warning when these are used, but will allow them to be used.
  • donkeycar/parts/actuator.py:
    • Added PulseController() as a general pwm controller that uses pin provider for pwm source. This can replace the prior pulse controllers; PCA9685, PiGPIO_PWM, and RPi_GPIO_Servo and eliminate the slight differences those various pulse controllers have.
    • Replaced L298N_HBridge_DC_Motor which was dedicated to RPi.GPIO with L298N_HBridge_3pin which does the same thing but uses pin provider so that it can be used on Jetson Nano (or with a PCA9685). I can test this configuration.
    • Replaced Mini_HBridge_DC_Motor_PWM which was dedicated to RPi.GPIO with L298N_HBridge_2pin which does the same thing but uses pin provider so that it can be used on Jetson Nano (or with a PCA9685). This has been tested with a logic analyzer, but it would be good to get this tested with a real robot that uses an L298N Mini HBridge controller.
    • Added methods to calculate 12bit pulse values from pulse length in microseconds and visa versa
    • Marked a lot of stuff as deprecated and commented those things. Some of those things we may want to keep and integrate with the framework better. This is where I need a lot of input from those individuals that added this code; is this worth keeping in this repository. If so, should it be better integrated so that it is part of a template and has configuration defaults and documentation? If not, do we move this out into it's own file and add comments on how to use it. Do we just remove it?
  • donkeycar/templates/complete.py/cfg_complete.py
    • added the PWM_STEERING_THROTTLE drivetrain configuration which uses the PulseController. This is a drivetrain for RC cars with Servo/ESC. It uses pin provider to specify the pwm pins. The idea is that in version 5.x, we would remove the other Servo/ESC configurations and just have this one. I can test this, but since this the Servo/ESC car is the majority of what is out there, it would be good to get others to test this.
    • modified the DC_TWO_WHEEL and DC_TWO_WHEEL_L298N drivetrains to use the pin provider versions of the drivers. This is a place I need input; this replaces the old drivers and configuration such that users of the prior configuration will be forced to update their pin definitions. Is this OK or should I keep the old code and deprecate it like I have done elsewhere. I think this is not used very much, so the impact would be small, but I don't have data to support that hunch.
    • made DRIVE_TRAIN_TYPE = "PWM_STEERING_THROTTLE" the default drivetrain configuration so that newly created cars will use the new pin provider drivers. This would not affect older configurations, just newly created configurations. It would be a way of getting a lot more real world testing on this prior to making this permanent in 5.x
  • donkeycar/templates/calibrate.py and donkeycar/parts/web_controller/web.py and donkeycar/parts/web_controller/templates/calibrate.html: updated web calibration to support DRIVE_TRAIN_TYPE = "PWM_STEERING_THROTTLE". I could use help testing this.
  • donkeycar/management/base.py : added support to the donkey calibrate command for pin provider. I've tested this with a PCA9685 servo/esc setup. I could use help testing other setups.
  • I added or updated a bunch of unit tests.

Verification before merging:

  • Test Plan passes for all configurations
  • Code branch PR (this PR) is approved
  • Corresponding Documentation branch PR is approved.

@Ezward Ezward changed the title 945 refactor motor drivers DRAFT 945 refactor motor drivers Nov 3, 2021
@TCIII
Copy link
Contributor

TCIII commented Nov 6, 2021

@Ezward,

Does not run on a NVIDIA Nano/Xavier as pins.py attempts to import RPi.GPIO as GPIO on a Nano/Xavier.
Pin.py needs to detect the SBC in use so that either Rpi.GPIO can be loaded for the Rpi or Jetson.GPIO (Jetson.GPIO comes preinstalled on Jetpack) can be loaded for the Nano/Xavier.

@Ezward
Copy link
Contributor Author

Ezward commented Nov 10, 2021

@Ezward,

Does not run on a NVIDIA Nano/Xavier as pins.py attempts to import RPi.GPIO as GPIO on a Nano/Xavier. Pin.py needs to detect the SBC in use so that either Rpi.GPIO can be loaded for the Rpi or Jetson.GPIO (Jetson.GPIO comes preinstalled on Jetpack) can be loaded for the Nano/Xavier.

That is a bug in dev branch as well. I've fixed it and pushed change to this branch.

Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ezward - I think this is a great change, only minor comments / questions. In general I observed, that you don't use f-strings or logging and your line lengths are not generally pep-8 compliant. Maybe you can change that?

donkeycar/management/base.py Outdated Show resolved Hide resolved
donkeycar/parts/pins.py Outdated Show resolved Hide resolved
donkeycar/parts/pins.py Outdated Show resolved Hide resolved
donkeycar/parts/pins.py Outdated Show resolved Hide resolved
mix pin schemes.
"""
prev_scheme = GPIO.getmode() or pin_scheme
GPIO.setmode(pin_scheme)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an observation here - our PWM actuator parts are multithreaded for performance reasons so they don't clog up the vehicle loop. I hope there is no issue if these parts will talk to the actuators from different threads.

Copy link
Contributor Author

@Ezward Ezward Nov 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I don't like this pattern where GPIO is a singleton with shared state. I agree this could be an issue if there are pins using different schemes. One approach would be to require a global scheme to be set then just check against it and runtime fail in pin spec does not match. Another approach is to use a semaphore/lock to make sure this section is not accessed by more than one thread.


def start(self, state:int=PinState.LOW) -> None:
if self.state() != PinState.NOT_STARTED:
raise RuntimeError("Attempt to start OutputPin that is already started.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add the PIN number to the exception message, so we know which ones is causing a problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...? not in this version.. maybe still on your local change...

donkeycar/parts/pins.py Show resolved Hide resolved
donkeycar/templates/calibrate.py Show resolved Hide resolved
return new_func2

else:
raise TypeError(repr(type(reason)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing newline

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks.

@Ezward Ezward self-assigned this Nov 26, 2021
@Ezward Ezward requested a review from DocGarbanzo November 26, 2021 21:41
try:
self.pwm.set_pwm(channel, 0, pulse)
except:
self.pwm.set_pwm(channel, 0, pulse)
Copy link
Contributor

@DocGarbanzo DocGarbanzo Nov 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks spurious, if the try failed we should log an error, but not call again w/o try/except.

Copy link
Contributor Author

@Ezward Ezward Nov 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ported this (it was already in our code). Looks like a one-time retry. I'll check with the original author why it is there, but I don't want to remove it without understanding that. I can't really hurt leaving it, but it might hurt taking it out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was Doug in d1efdfa, and the code was in the PCA9685 before. I have taken this bit out of my fork a long time ago, as I saw regular issues where it was throwing from the except branch, because the I2C can sometimes produce a hick up. If a single pwm signal gets missed, we are just a missing a control for 1/vehicle_HZ seconds, so not too bad.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DocGarbanzo Is there a reason not to do this retry?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if there is an I2C error, which is something I have observed on my car, then the non-exception handled retry might throw which will stop the vehicle loop. The except code should simply log an error, and when the vehicle loop comes around again, it will try to set it again.

pulse = int(4096 * duty_cycle)
try:
self.pwm.set_pwm(self.channel, 0, pulse)
except:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This issue is still here....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I"m ok with the except: clause doing a retry.

@Ezward Ezward changed the title DRAFT 945 refactor motor drivers 945 refactor motor drivers Dec 23, 2021
@Ezward Ezward requested a review from DocGarbanzo December 23, 2021 01:07
self.pwm.set_pwm(channel, 0, pulse)
except:
self.pwm.set_pwm(channel, 0, pulse)
self.pwm.set_pwm(channel, 0, pulse)
Copy link
Contributor

@DocGarbanzo DocGarbanzo Dec 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this we should keep the try/catch and incorporate proper exception handling like:

try:
    self.pwm.set_pwm(channel, 0, pulse)
except Exception as e:
    logger.error(f'Error in talking to PCA9685 channel {channel}: {str(e)}')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks.

Ezward added 17 commits January 3, 2022 11:01
- abstract classes  InputPin, OutputPin and PwmPin
  define interface for pin types.
- input_pin_by_id(), output_pin_by_id(), pwm_pin_by_id()
  are factories for creating pins using various
  underlying technologies.
- the point is to decouple the control of things like
  motors or encoders from a particular pin library,
  so we don't have to create a whole new redundant
  part if we want to use a different set of hardward.
- For instance, our L298N motor driver parts use
  RPi.GPIO for enable pins and pwm pins.  This means
  if you have a robot that uses a PCA9685 to control
  the enable and pwm for your L298N controller, you
  have to write a whole new motor driver to use it.
- This change makes it so we can make our parts that
  use ttl and pwm pins more reusable.
- this motor driver can use Rpi.GPIO or PCA9685 pins
  for enable and/or pwm pins, so we can use one
  driver for different pin technologies.
- This will even allow a mix of underlying
  pin providers for a single motor.
- For instance, the DuckieBot robots use Jetson Nano
  gpio pins for enable, but they use PCA9685
  to generate pwm signals.  With this new motor
  driver we can make Donkeycar work on that robot.
- donkeycar.parts.pins has high level pin api
- also made some fixes that I found when testing
- currently it appears InputPinGpio and OutputPinGpio
  function correctly, but PwmPinGpio does not
- needed to call pwm.start(duty) to start duty cycle
- older code used 1 to 100 duty cycle value and so did
  some truncation to integer.  That left over code
  cause throttles to be truncated to -1 or 0.
- added pin-provider configuration for mini-hbridge
- lots of commenting so folks can understand the different
  motor drivers, how to wire them and how to configure them.
- Since DC_STEER_THROTTLE used the mini-hbridge driver
  it can use the new versino that uses pin providers.
- update the config in cfg_complete.py as well
- This then makes the DC_STEER_THROTTLE, DC_TWO_WHEEL, DC_TWO_WHEEL_L298N
  drivers support PCA9685, Rpi.GPIO and pigpio pin providers
- added @deprecated decorator to mark classes and
  functions that are to be removed in the future
- Added PulseController that uses pin provider api
  to provide the set_pulse() api needed by some
  steering/throttle controllers.
- Marked some other set_pulse() controllers
  that are now redundant as @deprecated
- added a lot of commenting on those classes that
  are deprecated, including why and how we may
  move ahead if we wish to keep the functionality.
  Some of these fall into code that is not actually
  integrated into a template and is not documented
  either in docs.donkeycar.com or the code.
  Some of this code, like the Arduino Firmata, could
  be usefully integrated into the pins.py pin provider
  api but others would be redundant or otherwise
  unnecessary.
Ezward and others added 13 commits January 3, 2022 11:05
update install dependencies for nano

nano was installed RPi.GPIO, but needs Jetson.GPIO.
              'Jetson.GPIO',
              'pyserial',
- 'speed' not defined; should be 'throttle'
- use actuator.L298N_HBridge_2pin for both motors
- SERVO_HBRIDGE_2PIN supports steering servo and
  an HBridge motor driver using 2 pwm pins for motor controller
- SERVO_HBRIDGE_3PIN supports steering servo and
  an HBridge motor driver with 2 ttl pins for direction
  and 1 pwm for duty cycle to control the motor.
- NOTE: I'm using a 120 character line
Fixed lint (Flake8) issues, update class/method docs
- using 120 character line length
- prior code would try to change pin scheme per pin, however
  RPi.GPIO will only let pin scheme to be set once.
- code now makes sure that pin scheme is set and does not change.
- the except clause did not try to differentiate the
  exception so the retry logic might not be the right
  thing to do there.
- Now any exception will leak and we will actually see it
  so we can write appropriate code in the future if
  we see that happening.
@Ezward Ezward force-pushed the 945-refactor-motor-drivers branch from c5ef9d6 to fba7766 Compare January 3, 2022 19:23
@Ezward
Copy link
Contributor Author

Ezward commented Jan 3, 2022

I've rebased against dev and updated the RC hat docs to reference the new drive train type. I've completed most of the tests in #945 (comment) . Those that are not completed are because of lack of hardware. However, each of those have minimal risk because we have tested aspects of those configurations when we tested other configurations. @DocGarbanzo I think this is good to go if you do.

@Ezward Ezward requested a review from DocGarbanzo January 3, 2022 20:44
@DocGarbanzo
Copy link
Contributor

I hope you have re-based against dev - can you confirm? Can you also bump the version, that helps if folks come with questions and we see exactly which version of dev they are using.

Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good to me!

donkeycar/parts/pins.py Show resolved Hide resolved
donkeycar/management/base.py Show resolved Hide resolved
@Ezward
Copy link
Contributor Author

Ezward commented Jan 3, 2022

@DocGarbanzo Yes, I rebased against dev (NOT master, sorry. I've fixed the comment). I've bumped version to version="4.3.3". Thanks for all of your input on this complex branch.

@Ezward Ezward requested a review from DocGarbanzo January 3, 2022 21:04
Copy link
Contributor

@DocGarbanzo DocGarbanzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome.

@DocGarbanzo DocGarbanzo merged commit 3c03a54 into dev Jan 4, 2022
@Ezward Ezward deleted the 945-refactor-motor-drivers branch January 5, 2022 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants