Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IPinIP tunnel table full error #21516

Closed
sdszhang opened this issue Jan 23, 2025 · 1 comment
Closed

IPinIP tunnel table full error #21516

sdszhang opened this issue Jan 23, 2025 · 1 comment
Labels
Triaged this issue has been triaged

Comments

@sdszhang
Copy link
Contributor

sdszhang commented Jan 23, 2025 •
edited
Loading

When there are a lot of interfaces in the system, SAI report IPinIP table full. This is seen more system with 200+ interfaces.

2025 Jan  8 00:54:15.857936 tb-1 NOTICE swss#orchagent: :- addDecapTunnelTermEntry: Created tunnel decap term entry fc00::71.
2025 Jan  8 00:54:15.859186 tb-1 ERR syncd#syncd: [none] SAI_API_TUNNEL:mptnl_xgs_ipinip_create_terminator:564 SDK bcm_tunnel_terminator_add:  sip(0xfc)
2025 Jan  8 00:54:15.859186 tb-1 ERR syncd#syncd: [none] SAI_API_TUNNEL:mptnl_xgs_ipinip_create_terminator:567 bcm_tunnel_terminator_add failed with error Table full (0xfffffffa).
2025 Jan  8 00:54:15.859212 tb-1 ERR syncd#syncd: [none] SAI_API_TUNNEL:mptnl_xgs_flexflow_create_terminator:1688 mptnl_xgs_ipinip_create_terminator failed with error -13.
2025 Jan  8 00:54:15.859212 tb-1 ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_mptnl_terminator_create:2739 mptnl_hw_create_terminator failed with error -13.
2025 Jan  8 00:54:15.859392 tb-1 ERR syncd#syncd: [none] SAI_API_TUNNEL:brcm_sai_tnl_mp_create_tunnel_term_table_entry:5273 _brcm_sai_mptnl_terminator_create failed with error -13.
2025 Jan  8 00:54:15.859686 tb-1 ERR swss#orchagent: :- create: create status: SAI_STATUS_TABLE_FULL
2025 Jan  8 00:54:15.859732 tb-1 ERR swss#orchagent: :- addDecapTunnelTermEntry: Failed to create tunnel decap term entry fc00::75/128.
2025 Jan  8 00:54:15.859785 tb-1 ERR swss#orchagent: :- handleSaiCreateStatus: Encountered failure in create operation, exiting orchagent, SAI API: SAI_API_TUNNEL, status: SAI_STATUS_TABLE_FULL
2025 Jan  8 00:54:15.859808 tb-1 ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_TABLE_FULL
2025 Jan  8 00:54:15.859847 tb-1 ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_VR_ID: oid:0x30000000000eb
2025 Jan  8 00:54:15.859847 tb-1 NOTICE swss#orchagent: :- notifySyncd: sending syncd: SYNCD_INVOKE_DUMP
2025 Jan  8 00:54:15.859847 tb-1 ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_TYPE: SAI_TUNNEL_TERM_TABLE_ENTRY_TYPE_P2MP
2025 Jan  8 00:54:15.859861 tb-1 ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_TUNNEL_TYPE: SAI_TUNNEL_TYPE_IPINIP
2025 Jan  8 00:54:15.859861 tb-1 ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_ACTION_TUNNEL_ID: oid:0x2a000000002961
2025 Jan  8 00:54:15.859873 tb-1 ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_TERM_TABLE_ENTRY_ATTR_DST_IP: fc00::75
2025 Jan  8 00:54:15.860370 tb-1 NOTICE syncd#syncd: :- processNotifySyncd: Invoking SAI failure dump
2025 Jan  8 00:54:15.869028 tb-1 NOTICE swss#orchagent: :- sai_redis_notify_syncd: invoked DUMP succeeded
@sdszhang sdszhang changed the title IPinIP tunnel limitation IPinIP tunnel table full error Jan 23, 2025
@sdszhang sdszhang mentioned this issue Jan 23, 2025
Merged
2 tasks
@vdahiya12
Copy link
Contributor

workaround fix is merged

@vdahiya12 vdahiya12 added the Triaged this issue has been triaged label Jan 29, 2025
stepanblyschak pushed a commit to stepanblyschak/sonic-buildimage that referenced this issue Jan 31, 2025
@mssonicbld
skip ipinip tunnel creation if many interfaces (sonic-net#558)
3548e0c 
<!--
 Please make sure you've read and understood our contributing guidelines:
 https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md

 failure_prs.log Make sure all your commits include a signature generated with `git commit -s` **

 If this is a bug fix, make sure your description includes "fixes #xxxx", or
 "closes #xxxx" or "resolves #xxxx"

 Please provide the following information:
-->

#### Why I did it
Temporary workaround for sonic-net#21516. Issue is seen with 200+ interface. So, used >128 as condition instead of >256.

##### Work item tracking
- Microsoft ADO **(number only)**:

#### How I did it
Reset the address list if port count is over 128. So no more IPv4/IPv6 IPinIP tunnel will be created for interfaces.
tunnel interfaces on loopback and vlan MP2MP are still there.

#### How to verify it
Loaded the image with more than 200 interfaces, not seeing TABLE_FULL issue anymore.

<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->

#### Which release branch to backport (provide reason below if selected)

<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->

#### Tested branch (Please provide the tested image version)

<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->

- [ ] <!-- image version 1 -->
- [ ] <!-- image version 2 -->

#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->

<!--
 Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->

#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->

#### A picture of a cute animal (not mandatory but encouraged)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Triaged this issue has been triaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vdahiya12 @sdszhang