Random Test Failure: Faker::Omniauth tests fail when generating a name with an apostrophe

[alextaujenis]

Describe The Bug

The Faker::Omniauth module sometimes initializes a last name with an apostrophe (such as O'Connell), which will generate an email address that fails the validation tests within TestFakerInternetOmniauth.

To Reproduce

With Faker v3.3.1:

1. Locate the test file test/faker/default/test_faker_omniauth.rb.
2. Provide a name with an apostrophe such as: Faker::Omniauth.linkedin(name: "Alexis O'Connell")
3. Validate the returned omniauth email address with def email_regex(first_name, last_name) (found within the test file, also below in the reproduction script).

Use the reproduction script below to reproduce the issue:

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem 'faker', :git => 'https://github.com/faker-ruby/faker.git', :branch => 'main'
  gem "minitest"
end

require "minitest/autorun"

class TestFakerInternetOmniauth < Minitest::Test
  def test_omniauth_linkedin
    @tester = Faker::Omniauth
    auth = @tester.linkedin(name: "Alexis O'Connell")
    info = auth[:info]
    first_name = info[:first_name].downcase
    last_name = info[:last_name].downcase

    assert_match email_regex(first_name, last_name), info[:email]
  end

  def email_regex(first_name, last_name)
    /(#{first_name}(.|_)#{last_name}|#{last_name}(.|_)#{first_name})@(.*).(example|test)/i
  end
end

Expected Behavior

The TestFakerInternetOmniauth tests should pass when generating an omniauth profile that includes an apostrophe in the name.

Additional Context

This email validation issue was mentioned in PR #2940 (review).

[keshavbiswa]

So currently when we run

Faker::Internet.email(name: "Alexis O'MConnell")

#=> o_alexis_mconnell@zieme.example

This might not be what we expect. But I'm not sure what is the correct representation for this?

[keshavbiswa]

Is it o.connell_alexis@zieme.example?

[alextaujenis]

@keshavbiswa exactly...

So this generator will return all of these possible permutations of the username:

alexis.o.connell@domain.example
alexis.connell.o@domain.example
connell.alexis.o@domain.example
connell.o.alexis@domain.example
o.connell.alexis@domain.example
o.alexis.connell@domain.example
alexis_o_connell@domain.example
alexis_connell_o@domain.example
connell_alexis_o@domain.example
connell_o_alexis@domain.example
o_connell_alexis@domain.example
o_alexis_connell@domain.example

and the test regex is only validating these possibilities:

alexis.o'connell@domain.example
o'connell.alexis@domain.example
alexis_o'connell@domain.example
o'connell_alexis@domain.example

Here is the relevant code that generates these values within Faker::Internet.email (code edited below for brevity):

  def email(name: nil, separators: nil, domain: nil)
    local_part = username(specifier: name, separators: separators)
    generate_domain = domain_name(domain: domain)
    return [local_part, generate_domain].join('@')
  end

  def username(specifier: nil, separators: %w[. _])
    return shuffle(specifier.scan(/[[:word:]]+/)).join(sample(separators)).downcase
  end

What's happening here is that Faker::Internet.email will accept a name and split the string attempting to find the different parts, shuffle the different parts, then join them back together with a random separator of a period or underscore. The issue is the apostrophe in O'Connell being treated as a word separator and the O is being independently shuffled as part of the username.

This issue has what appears to be several cascading problems that could be solved different ways:

1. An apostrophe is valid within an email address, so this generator could be adjusted to keep O'Connell together and not shuffle the O within the username, then adjust the email_regex to accommodate. This would allow the test to still verify the provided username and returned email address without stripping the apostrophe.

2. The email_regex could be adjusted to just verify ANY valid email address instead of all possible first.last@domain.example permutations. However, this would reduce the test suites ability to detect the expected outcome of including the user's full name within the generated email address.

3. Strip the apostrophe from the last name within the test suite and just return permutations of alexis.oconnell@domain.example. This will still shuffle last names with an apostrophe when they are provided in real-world scenarios.

[keshavbiswa]

@keshavbiswa exactly...

So this generator will return all of these possible permutations of the username:

alexis.o.connell@domain.example
alexis.connell.o@domain.example
connell.alexis.o@domain.example
connell.o.alexis@domain.example
o.connell.alexis@domain.example
o.alexis.connell@domain.example
alexis_o_connell@domain.example
alexis_connell_o@domain.example
connell_alexis_o@domain.example
connell_o_alexis@domain.example
o_connell_alexis@domain.example
o_alexis_connell@domain.example

and the test regex is only validating these possibilities:

alexis.o'connell@domain.example
o'connell.alexis@domain.example
alexis_o'connell@domain.example
o'connell_alexis@domain.example

Here is the relevant code that generates these values within Faker::Internet.email (code edited below for brevity):

  def email(name: nil, separators: nil, domain: nil)
    local_part = username(specifier: name, separators: separators)
    generate_domain = domain_name(domain: domain)
    return [local_part, generate_domain].join('@')
  end

  def username(specifier: nil, separators: %w[. _])
    return shuffle(specifier.scan(/[[:word:]]+/)).join(sample(separators)).downcase
  end

What's happening here is that Faker::Internet.email will accept a name and split the string attempting to find the different parts, shuffle the different parts, then join them back together with a random separator of a period or underscore. The issue is the apostrophe in O'Connell being treated as a word separator and the O is being independently shuffled as part of the username.

This issue has what appears to be several cascading problems that could be solved different ways:

1. An apostrophe is valid within an email address, so this generator could be adjusted to keep O'Connell together and not shuffle the O within the username, then adjust the email_regex to accommodate. This would allow the test to still verify the provided username and returned email address without stripping the apostrophe.
2. The email_regex could be adjusted to just verify ANY valid email address instead of all possible first.last@domain.example permutations. However, this would reduce the test suites ability to detect the expected outcome of including the user's full name within the generated email address.
3. Strip the apostrophe from the last name within the test suite and just return permutations of alexis.oconnell@domain.example. This will still shuffle last names with an apostrophe when they are provided in real-world scenarios.

I think I'll try the 3rd approach, although apostrophe's are valid, it might still look weird for others, so maybe it's best to keep it simple? Also like you mentioned, it might not be a good idea to verify just any valid email address.

[keshavbiswa]

On second thought, I feel even the 3rd approach might not be a good idea since we're not covering all the scenarios. Maybe we keep the apostrophe together, do we want o_alexis_connell@domain.example, seems incorrect to me.

[stefannibrasil]

Oh, this is an interesting issue. An apostrophe is not a valid character per Google so I don't think it makes sense keeping it around for the email.

I think it makes sense to just drop the apostrophe when generating the email address and merge it into the last name. Although we could use ., I don't think it would make sense to split it out of the last name. So dropping it and merging it with the last name seems to be the simplest approach to take. So the only permutations would be:

alexis_oconnell@domain.example
oconnell_alexis@domain.example

[alextaujenis]

@stefannibrasil It is completely valid to include an apostrophe within an email address and I'd suggest keeping this character within the Faker email generator. I'd suggest that it's important to guide these decisions based upon the IETF RFC standards.

If you strip out the apostrophe you will still have to accommodate for random test failures when checking with the current Omniauth regex:

  def email_regex(first_name, last_name)
    /(#{first_name}(.|_)#{last_name}|#{last_name}(.|_)#{first_name})@(.*).(example|test)/i
  end

You can get around this issue by also stripping apostrophes from the first and last names within the test data, but generally speaking I think it's an easier fix to include the apostrophe instead of stripping it. Keeping the apostrophe and fixing the broken test seems to be as simple as replacing specifier.scan(/[[:word:]]+/) with specifier.split within the Faker::Internet.username method.

[stefannibrasil]

Ah, thank you! I was mainly concerned about separating the apostrophe from the last name but confirmed that is handled in the linked PR.