Does one profile allow multiple fanout? #12

snakwu · 2020-03-29T16:26:16Z

Does one profile allow multiple fanout?
Used for dns shunt parallel query

edwarnicke · 2020-03-29T18:06:59Z

@snakwu Could you say more about what you are asking? If I'm reading your question correctly the answer is yes...but I'm not sure I'm reading it correctly :)

@denis-tingajkin

snakwu · 2020-03-29T18:08:49Z

@snakwu Could you say more about what you are asking? If I'm reading your question correctly the answer is yes...but I'm not sure I'm reading it correctly :)

@denis-tingajkin

我一直关注你的这个插件，我想用来做dns分流，不知道一个区域，可用多次这个插件吗？

snakwu · 2020-03-29T18:16:48Z

@snakwu Could you say more about what you are asking? If I'm reading your question correctly the answer is yes...but I'm not sure I'm reading it correctly :)

@denis-tingajkin

我一直关注你的这个插件，我想用来做dns分流，不知道一个区域，可用多次这个插件吗？

还有，except参数，可否改用map查询列表方式来排除，except列表我填了7万多个域名

edwarnicke · 2020-03-29T19:34:48Z

@snakwu apologies... I cannot read chinese

snakwu · 2020-03-29T19:56:17Z

@snakwu apologies... I cannot read chinese

online translation?

denis-tingaikin · 2020-03-30T03:34:25Z

Hello @snakwu,

Does one profile allow multiple fanout?

If you meant something like this

. {
    fanout . {IP1} {IP2} {
        except domain-a domain-b domain-c 
    }
} 
domain-b {
    fanout . {IP5} {IP6}
}
domain-a {
    fanout{IP3} {IP4} {
        except domain-b
    }
}

then yes, it does.

except列表我填了7万多个域名

Interesting case... I'm not sure that it will work fast because of expect directive handling linear(it was not designed for a ton of labels).

Did you ask Coredns about the case with 70 000 ignore domains? Probably they have some plugin for ignoring a ton of domains.

snakwu · 2020-03-30T04:02:09Z

Hello @snakwu,

Does one profile allow multiple fanout?

If you meant something like this
. {
    fanout . {IP1} {IP2} {
        except domain-a domain-b domain-c 
    }
    fanout domain-a {IP3} {IP4} {
        except domain-b domain-c
    }
    fanout . {IP5} {IP6}
}
then yes, it does.

except列表我填了7万多个域名

Interesting case... I'm not sure that it will work fast because of expect directive handling linear(it was not designed for a ton of labels).

Did you ask Coredns about the case with 70 000 ignore domains? Probably they have some plugin for ignoring a ton of domains.

coredns需说不能超16个，但我填了7万个也一样工作，但怎才让他更好的工作是个问题

snakwu · 2020-03-30T04:11:36Z

Hello @snakwu,

Does one profile allow multiple fanout?

If you meant something like this
. {
    fanout . {IP1} {IP2} {
        except domain-a domain-b domain-c 
    }
    fanout domain-a {IP3} {IP4} {
        except domain-b domain-c
    }
    fanout . {IP5} {IP6}
}
then yes, it does.

except列表我填了7万多个域名

Interesting case... I'm not sure that it will work fast because of expect directive handling linear(it was not designed for a ton of labels).

Did you ask Coredns about the case with 70 000 ignore domains? Probably they have some plugin for ignoring a ton of domains.

. {
fanout . {IP1} {IP2} {
except domain-a domain-b domain-c
}
fanout . {IP3} {IP4}
}
cache

这样配置可以吗，我想要的是不在except列表的用IP1和IP2查询，如果在列表的就跳转到下个fanout用IP3和IP4查询

denis-tingaikin · 2020-03-30T04:16:35Z

coredns需说不能超16个，但我填了7万个也一样工作，但怎才让他更好的工作是个问题

Probably we can improve except directive to allow it to load entries from file... Also need to think about how to store these entries in memory to have fast answers about is domain ignored. If you have some ideas you can prepare a patch.

There are two entry points if you want to patch it:

https://github.com/networkservicemesh/fanout/blob/master/setup.go#L162 //add somehow loading from the file here
https://github.com/networkservicemesh/fanout/blob/master/fanout.go#L141 //somehow smart handle a ton of ignoring domains(for example it should work fast for 70 000 entries)

这样配置可以吗，我想要的是不在except列表的用IP1和IP2查询，如果在列表的就跳转到下个fanout用IP3和IP4查询

Yes, in general, it should work. Let us know if it will not work :)

snakwu · 2020-03-30T04:19:59Z

coredns需说不能超16个，但我填了7万个也一样工作，但怎才让他更好的工作是个问题

Probably we can improve except directive to allow it to load entries from file... Also need to think about how to store these entries in memory to have fast answers about is domain ignored. If you have some ideas you can prepare a patch.

There are two entry points if you want to patch it:

https://github.com/networkservicemesh/fanout/blob/master/setup.go#L162 //add somehow loading from the file here

https://github.com/networkservicemesh/fanout/blob/master/fanout.go#L141 //somehow smart handle a ton of ignoring domains(for example it should work fast for 70 000 entries)

这样配置可以吗，我想要的是不在except列表的用IP1和IP2查询，如果在列表的就跳转到下个fanout用IP3和IP4查询

Yes, in general, it should work. Let us know if it will not work :)

Thank you very much！如果说用map的算法是否能更有效的工作？其体我也不懂

snakwu · 2020-03-30T04:36:36Z

coredns需说不能超16个，但我填了7万个也一样工作，但怎才让他更好的工作是个问题

Probably we can improve except directive to allow it to load entries from file... Also need to think about how to store these entries in memory to have fast answers about is domain ignored. If you have some ideas you can prepare a patch.

There are two entry points if you want to patch it:

https://github.com/networkservicemesh/fanout/blob/master/setup.go#L162 //add somehow loading from the file here

https://github.com/networkservicemesh/fanout/blob/master/fanout.go#L141 //somehow smart handle a ton of ignoring domains(for example it should work fast for 70 000 entries)

这样配置可以吗，我想要的是不在except列表的用IP1和IP2查询，如果在列表的就跳转到下个fanout用IP3和IP4查询

Yes, in general, it should work. Let us know if it will not work :)

. {
fanout . {IP1} {IP2} {
except domain-a domain-b domain-c
}
fanout . {IP3} {IP4}
}

is not work!
plugin/fanout: this plugin can only be used once per Server Block��

denis-tingaikin · 2020-03-30T04:51:22Z

如果说用map的算法是否能更有效的工作？

Not sure that we can just use a map there. I think first we need to check how works this method https://github.com/coredns/coredns/blob/master/plugin/normalize.go#L21.

is not work!
plugin/fanout: this plugin can only be used once per Server Block

Oops, I forgot that we have this check.

Comment above is updated

snakwu · 2020-03-30T05:29:31Z

如果说用map的算法是否能更有效的工作？

Not sure that we can just use a map there. I think first we need to check how works this method https://github.com/coredns/coredns/blob/master/plugin/normalize.go#L21.

is not work!
plugin/fanout: this plugin can only be used once per Server Block

Oops, I forgot that we have this check.

Comment above is updated

据我了解到，有人修改了forward实现map算法，请参考：
https://github.com/missdeer/coredns_custom_build/blob/master/forward-setup.go.patch

https://github.com/missdeer/coredns_custom_build/blob/master/forward.go.patch

denis-tingaikin · 2020-03-30T05:40:59Z

Interesting, then you can just apply this patch for fanout plugin. Also, you need to add an option to load ton entries from the file. I suggest adding here https://github.com/networkservicemesh/fanout/blob/master/setup.go#L155 case except-file.

snakwu · 2020-03-30T05:48:29Z

Interesting, then you can just apply this patch for fanout plugin. Also, you need to add an option to load ton entries from the file. I suggest adding here https://github.com/networkservicemesh/fanout/blob/master/setup.go#L155 case except-file.

soory!我对修改代码不是很熟悉，还得靠你们实现！非常感谢这么好的项目

denis-tingaikin · 2020-03-30T08:00:37Z

@edwarnicke Do you have any concerns related to this plugin fanout improvement?

edwarnicke · 2020-03-30T09:33:55Z

@denis-tingajkin I don't think so... it sounds like you are simply adding additional functionality, not changing core functionality... delighted that this is useful to others :)

snakwu · 2020-03-31T05:05:47Z

何时能实现map算法？

denis-tingaikin · 2020-03-31T13:15:33Z

@snakwu I'll discuss it today with the team.
Also, I've discussed with Coredns guys, they plan to add a specific plugin for your case. Can you check is this plugin suitable for you?

snakwu · 2020-03-31T13:22:54Z

@snakwu I'll discuss it today with the team.
Also, I've discussed with Coredns guys, they plan to add a specific plugin for your case. Can you check is this plugin suitable for you?

这个插件不合适我，我要实现的是在这个列表中的域名就跳转到下一个插件，用下一个插件来解释列表中的域名，block是直接返回空值！

denis-tingaikin · 2020-03-31T14:51:10Z

@fkautz can we spend time on this issue? It will not be used in NSM scenarios but can be used by fanout plugin users.

denis-tingaikin · 2020-04-09T07:23:10Z

@snakwu PR has been merged, can you please check is this problem still actual?

snakwu · 2020-04-09T07:24:40Z

@snakwu PR has been merged, can you please check is this problem still actual?

ok，马上测试，等了两天了，为什么每次合并都要等待那么久！

snakwu · 2020-04-09T08:02:15Z

@snakwu PR has been merged, can you please check is this problem still actual?

我填写了7万多8万条except，现在明显比之前快了！！

denis-tingaikin · 2020-04-09T09:08:15Z

我填写了7万多8万条except，现在明显比之前快了！！

@snakwu Sounds good. Can we close this issue?

snakwu · 2020-04-09T09:10:03Z

我填写了7万多8万条except，现在明显比之前快了！！

@snakwu Sounds good. Can we close this issue?

ok，这个map算法还是有效的，这个issue可以关了，有问题我再及时反馈，非常感谢你的付出！

denis-tingaikin · 2020-04-09T09:11:04Z

Fixed by #17

snakwu · 2020-04-09T09:11:32Z

我填写了7万多8万条except，现在明显比之前快了！！

@snakwu Sounds good. Can we close this issue?

但还有一个发现，就是在多个dns服务器中查询问题，往往反回的不是最快的ip查询结果，还麻烦你留意下这方面看可不可以优化一下，谢谢

snakwu · 2020-04-09T12:35:25Z

@denis-tingajkin
关于except-file的用法，这个文件格式应该是怎么样写这个，用except写在配置里面，不是以except-file单独一个文件，那个效率更高？

ps：如果except-file文件更新了，他对于reload有效吗？会重新加载吗？

denis-tingaikin self-assigned this Apr 6, 2020

denis-tingaikin added bug Something isn't working enhancement New feature or request and removed bug Something isn't working labels Apr 6, 2020

denis-tingaikin mentioned this issue Apr 7, 2020

Fanout plugin: Improve except option #17

Merged

denis-tingaikin closed this as completed Apr 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Does one profile allow multiple fanout? #12

Does one profile allow multiple fanout? #12

snakwu commented Mar 29, 2020

edwarnicke commented Mar 29, 2020

snakwu commented Mar 29, 2020

snakwu commented Mar 29, 2020

edwarnicke commented Mar 29, 2020

snakwu commented Mar 29, 2020

denis-tingaikin commented Mar 30, 2020 •

edited

Loading

snakwu commented Mar 30, 2020

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020

snakwu commented Mar 30, 2020

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020 •

edited

Loading

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020 •

edited

Loading

edwarnicke commented Mar 30, 2020

snakwu commented Mar 31, 2020

denis-tingaikin commented Mar 31, 2020

snakwu commented Mar 31, 2020 •

edited

Loading

denis-tingaikin commented Mar 31, 2020

denis-tingaikin commented Apr 9, 2020

snakwu commented Apr 9, 2020

snakwu commented Apr 9, 2020

denis-tingaikin commented Apr 9, 2020

snakwu commented Apr 9, 2020

denis-tingaikin commented Apr 9, 2020

snakwu commented Apr 9, 2020

snakwu commented Apr 9, 2020 •

edited

Loading

Does one profile allow multiple fanout? #12

Does one profile allow multiple fanout? #12

Comments

snakwu commented Mar 29, 2020

edwarnicke commented Mar 29, 2020

snakwu commented Mar 29, 2020

snakwu commented Mar 29, 2020

edwarnicke commented Mar 29, 2020

snakwu commented Mar 29, 2020

denis-tingaikin commented Mar 30, 2020 • edited Loading

snakwu commented Mar 30, 2020

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020

snakwu commented Mar 30, 2020

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020 • edited Loading

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020

snakwu commented Mar 30, 2020

denis-tingaikin commented Mar 30, 2020 • edited Loading

edwarnicke commented Mar 30, 2020

snakwu commented Mar 31, 2020

denis-tingaikin commented Mar 31, 2020

snakwu commented Mar 31, 2020 • edited Loading

denis-tingaikin commented Mar 31, 2020

denis-tingaikin commented Apr 9, 2020

snakwu commented Apr 9, 2020

snakwu commented Apr 9, 2020

denis-tingaikin commented Apr 9, 2020

snakwu commented Apr 9, 2020

denis-tingaikin commented Apr 9, 2020

snakwu commented Apr 9, 2020

snakwu commented Apr 9, 2020 • edited Loading

denis-tingaikin commented Mar 30, 2020 •

edited

Loading

denis-tingaikin commented Mar 30, 2020 •

edited

Loading

denis-tingaikin commented Mar 30, 2020 •

edited

Loading

snakwu commented Mar 31, 2020 •

edited

Loading

snakwu commented Apr 9, 2020 •

edited

Loading