Example of using Embedding

[abagher2]

Hi,

Thanks for creating this package. I am very happy to see a native ruby implementation for DNNs. I have been playing around with it to use in a machine learning class I am teaching.

I have a dataset with two different categorical features and I would like to use one Embedding for each feature. Some other features in my dataset are numeric. In other packages there is a way to slice along an axis, apply the embedding and the concatenate. I can't figure out how to do this. There is code for an Embedding layer but no examples.

Also, can you show how embed features into multiple dimensions.

class MyModel < Model 
 ##...
  def forward x
    e1 = @embed1.(x) # Slice
    e2 = @embed2.(x) # Slice
    x = Concatenate.(e1, e2, axis: 1)
    x = @dense.(x)
  end
end

My data looks like this:

48.0, 2.0595, 1.0, 2.0, 1.0, 0.0

where the last two features are to be embedded and the rest are numeric.

Thanks!

[unagiootoro]

Hi,

Thank you for use to this library.

Currently, ruby-dnn does not support Slice.
However, by giving multiple inputs to the model,
the it is possible to apply different embeddings to two different categorical features.
It can be accomplished with the following code.

class MyModel < Model 
 ##...
  def forward(inputs)
    x1, x2 = *inputs
    e1 = @embed1.(x1)
    e2 = @embed2.(x2)
    x = Concatenate.(e1, e2, axis: 1)
    x = @dense.(x)
  end
end

model = MyModel.new
model.train([input1, input2], output, epochs)

And, Embedding can not be applied to multi-dimensional features.
Therefore, after applying Embedding to a one-dimensional feature,
the you need to make it multi-dimensional with Reshape.

class MyModel < Model
 ##...
  def forward(x)
    e = @embed.(x)
    x = Reshape.(x, [32, 32, 3])
    x = @conv2d.(x)
  end
end

※I am not good at English and may be wrong.

[unagiootoro]

There is no Embedding example because it's hard to prepare a natural language dataset. This is something I hope to improve in the future.

[abagher2]

Thanks for your response. I will try that. On Apr 7, 2020, at 6:46 AM, unagiootoro <notifications@github.com> wrote:  Hi, Thank you for use to this library. Currently, ruby-dnn does not support Slice. However, by giving multiple inputs to the model, the it is possible to apply different embeddings to two different categorical features. It can be accomplished with the following code. class MyModel < Model ##... def forward(inputs) x1, x2 = *inputs e1 = @embed1.(x1) e2 = @embed2.(x2) x = Concatenate.(e1, e2, axis: 1) x = @Dense.(x) endend model = MyModel.new model.train([input1, input2], output, epochs) And, Embedding can not be applied to multi-dimensional features. Therefore, after applying Embedding to a one-dimensional feature, the you need to make it multi-dimensional with Reshape. class MyModel < Model ##... def forward(x) e = @embed.(x) x = Reshape.(x, [32, 32, 3]) x = @conv2d.(x) endend ※I am not good at English and may be wrong. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIC2TXLIOTWIFTPHFH55K73RLMVE3ANCNFSM4L64H7BA> .