Fix bug in tokenizer with nil source value

[bahar-p]

Resolves https://github.com/Shopify/shopify/issues/560829

Fixes the Tokenizer to handle source with null value gracefully. The longer explanation can be found in this comment as I was investigating a Bugsnag exception.

Tl;Dr
The Tokenizer isn't currently resilient to source with nil value. Liquid C does a graceful conversion here. This has been causing a lot of exception when we run liquid ruby and the asset value is nil. I followed the same pattern as Liquid C tokenizer to convert nil to empty string.

Test

I added a test and able to reproduce the exception with the test (before adding the fix).

Before

The new test fails with an exception. Tokenizer.new(nil).tokenize raises:

NoMethodError: undefined method `empty?' for nil
    lib/liquid/tokenizer.rb:31:in `tokenize'
    lib/liquid/tokenizer.rb:12:in `initialize'
    .....

After

The new test passes. Tokenizer.new(nil).tokenize returns [].

[bahar-p]

I'm not entirely sure why liquid_c does .to_s.to_str. 🤷🏼‍♀️

to_s should be sufficient, but let me know if I'm missing anything here.

[mmorissette]

Seems like there was a legit reason originally to use both to_s.to_str, see #1421 (comment)

Looks like @ggmichaelgo then removed the logic here #1775

[bahar-p]

Yeah the first PR didn't introduce a bug though. In the second PR @ggmichaelgo is addressing a different bug, where then removed it from the Tokenizer and doing the conversion in Template class. But that's not always the entry point to the Tokenizer. Note that in Core the templates are subclasses of Liquid::Document not Liquid::Template.

My question is more around doing double conversion to_s.to_str which I'm not sure I understand yet. I've double checked the documents and they're essentially alias of each other

From the docs

to_s → str
Returns self.

If called on a subclass of String, converts the receiver to a [String](https://devdocs.io/ruby~3/string) object.

Also aliased as: to_str
Returns self.

If called on a subclass of [String](https://devdocs.io/ruby~3/string), converts the receiver to a String object.

Alias for: to_s

I'm asking Ruby folks to see what I may be missing. Will address in follow up as soon as I hear back and learn that we legitimately need the double conversion.