Need for a method hex2bytes(Vector{UInt8})

[sambitdash]

Many a times hex encoded bytes are received as part of IO or network operations. And that will need to be converted to bytes for further processing. Current method expects a conversion to a form of AbstractString before proceeding further.

julia> b=[UInt8('a'),UInt8('b'),UInt8('c'),UInt8('d')]
4-element Array{UInt8,1}:
 0x61
 0x62
 0x63
 0x64

julia> b |> String |> hex2bytes
2-element Array{UInt8,1}:
 0xab
 0xcd

julia> b |> hex2bytes
ERROR: MethodError: no method matching hex2bytes(::Array{UInt8,1})
Closest candidates are:
  hex2bytes(::AbstractString) at strings/util.jl:445
Stacktrace:
 [1] |>(::Array{UInt8,1}, ::Base.#hex2bytes) at ./operators.jl:904

I feel the 3rd option may be helpful as some of the data IO operations can be substantial if the files are large and String conversion is an unnecessary operation.

Detailed discussion in the thread:

https://discourse.julialang.org/t/need-for-a-method-hex2bytes-vector-uint8/5209

[StefanKarpinski]

The hex2bytes and bytes2hex functions should likely be deleted or at least moved to a package. But I guess that given that we have these functions currently, we may as well add these methods – not sure what else they could mean.

[sambitdash]

There is a similar approach taken by another package under BioJulia.

https://github.com/bicycle1885/TranscodingStreams.jl

But, the intent is more of streams and not on primitive types.

However, it may also help to change the signature, as typically such methods are more like filters. The output array can be pre-allocated by caller.

hex2bytes(output::AbstractArray{UInt8}, input::AbstractArray{UInt8}, nInBytes) --> returns number of bytes converted into output array.

FileIO or network IO, you would read in chunks and pass to this method for conversion. That way reallocation of the output buffer may not be needed for next chunk of data read.