Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More performance improvements #4210

Merged
merged 48 commits into from
Mar 29, 2023
Merged
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
d8f0007
Tweak release doc
parrt Feb 20, 2023
681d46a
Update release docs (#4136)
ericvergnaud Feb 23, 2023
89de611
Provide JavaScript port of TokenStreamRewriter (#3560)
eirikur-grid Feb 26, 2023
bc5a76f
export token channel (#4143)
ericvergnaud Feb 26, 2023
60b4e27
added null | string union type (#4147)
sharknoon Feb 28, 2023
29de023
Limit use of Posix threads to Unix (#4140)
hs-apotell Mar 3, 2023
c455b7f
fix: Some code that should have been destined for /v4 was instead put…
Mar 5, 2023
e25d499
Remove redundant source code from go runtime (#4155)
Mar 5, 2023
8ac9c4b
README: Update required Java version (#4129)
Krzmbrzl Mar 5, 2023
60a8a6d
cmake: Fix output dir for Visual Studio generator and for when ANTLR_…
Leo1690 Mar 5, 2023
2f4f85b
Cpp: Remove code duplication (#3995)
Krzmbrzl Mar 5, 2023
21b27df
Changed Parser typings (#4149)
sharknoon Mar 8, 2023
d77766b
fix: Fixes the failing go runtime test suite which was missing the /v…
Mar 9, 2023
4ab43ac
present antlr before versioning (#4156)
ericvergnaud Mar 9, 2023
b0713e2
fix: Prevent use of labels such as start= from generating code that c…
Mar 9, 2023
223c8ae
Feature/gotestfix (#4168)
Mar 10, 2023
c466045
Feature/docclean Greatly improve the godoc comments in the runtime (#…
Mar 10, 2023
ccd0ed4
Feature/fixembedding (#4176)
Mar 13, 2023
7fb762e
fix incorrect prototype (#4184)
ericvergnaud Mar 16, 2023
2ad9ef9
fix: Fixes merge arrays to perform an actual comparison rather than c…
jimidle Mar 15, 2023
f91b21e
doc: Update TODO comment in prediction_context now that #3967 is fixed
jimidle Mar 15, 2023
4a364b2
fix: Unbeliveably, the PredictionContext cache did not work!
jimidle Mar 15, 2023
58121d5
feat: Fix up the parsing cache
jimidle Mar 15, 2023
28ef5b2
fix: Massive performance improvement by fixing comparison functions
jimidle Mar 16, 2023
d0e8f2b
fix: delete commented out code
jimidle Mar 16, 2023
c98ffad
test: Tewmporarilly disable one of the tests programs for the go runtime
jimidle Mar 16, 2023
bf37980
feat: Improve the runtime performance of the go test suite
jimidle Mar 15, 2023
a597aec
fix: Minor updates to address Ivan's PR review
jimidle Mar 16, 2023
76972cb
fix: reverse accidental IntelliJ formatting
jimidle Mar 17, 2023
c4a4442
fix: Change codegen and runtime to eliminate panic() and recover() as…
jimidle Mar 17, 2023
c6bbbd3
feat: Another 50%+ performance improvement
jimidle Mar 18, 2023
76bb3e1
[Typescript] Add missing methods typing declaration (#4145)
hieunguyen2211 Mar 22, 2023
91e2909
do not modify String.prototype in js package (#4200)
jharris4 Mar 22, 2023
f1b8982
Issue #4185: Too many artifacts fail to upload (#4186)
hs-apotell Mar 22, 2023
3a62248
feat: Stop using pointers for INterval class
jimidle Mar 19, 2023
fc38127
feat: Refactor PredictionContexts so that it does not use an interface
jimidle Mar 20, 2023
bb5d674
feat: Remove ocde and structs that are now replaced
jimidle Mar 20, 2023
4196a07
feat: Incorporate predefined variables for empty cache into the one p…
jimidle Mar 20, 2023
7e60bb9
fix: Tiny correct to parent check for singleton context
jimidle Mar 24, 2023
2f396b3
fix: Correct config cache comparators improvie performance
jimidle Mar 28, 2023
84c22a8
fix: Correct comparison functions for PredictionContexts not they are…
jimidle Mar 28, 2023
7a2c3c3
fix: Properly fix the context caching so that it exactly relfects Java
jimidle Mar 28, 2023
62d8439
feat: Stop using interfaces for ATNConfigSets
jimidle Mar 28, 2023
4476b9b
fix: Remove useless Getters and Setters
jimidle Mar 28, 2023
18b5bbf
fix: Get rid of interface for ATNConfig
jimidle Mar 28, 2023
2e93c77
Merge branch 'dev' into feature/performance
Mar 28, 2023
3b923fa
fix: As suspected, merging on GitHub went slightly wrong.
jimidle Mar 28, 2023
86bc5cb
fix: remove new memory handler from this brnach
jimidle Mar 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: Refactor PredictionContexts so that it does not use an interface
Interfaces require two pointer lookups for functions and when used as
Generic, they require three. This change therefore yields a small
performance upgrade.

This change also corrects one or two copmarisons that were using pointer
comparison instead of Equals() and were bugs in the code I inherited.

Signed-off-by: Jim.Idle <jimi@idle.ws>
  • Loading branch information
jimidle committed Mar 28, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit fc381277c06c035543dc17c91b0733dbbab4190c
119 changes: 45 additions & 74 deletions runtime/Go/antlr/v4/array_prediction_context.go
Original file line number Diff line number Diff line change
@@ -1,40 +1,11 @@
package antlr

import (
"golang.org/x/exp/slices"
"strconv"
)

type ArrayPredictionContext struct {
BasePredictionContext
parents []PredictionContext
returnStates []int
}

func NewArrayPredictionContext(parents []PredictionContext, returnStates []int) *ArrayPredictionContext {
// Parent can be nil only if full ctx mode and we make an array
// from {@link //EMPTY} and non-empty. We merge {@link //EMPTY} by using
// nil parent and
// returnState == {@link //EmptyReturnState}.
hash := murmurInit(1)
for _, parent := range parents {
hash = murmurUpdate(hash, parent.Hash())
}
for _, returnState := range returnStates {
hash = murmurUpdate(hash, returnState)
}
hash = murmurFinish(hash, len(parents)<<1)

return &ArrayPredictionContext{
BasePredictionContext: BasePredictionContext{
cachedHash: hash,
pcType: PredictionContextArray,
},
parents: parents,
returnStates: returnStates,
}
}

func (a *ArrayPredictionContext) GetReturnStates() []int {
return a.returnStates
}
@@ -61,55 +32,55 @@ func (a *ArrayPredictionContext) getReturnState(index int) int {
return a.returnStates[index]
}

// Equals is the default comparison function for ArrayPredictionContext when no specialized
// implementation is needed for a collection
func (a *ArrayPredictionContext) Equals(o Collectable[PredictionContext]) bool {
if a == o {
return true
}
other, ok := o.(*ArrayPredictionContext)
if !ok {
return false
}
if a.cachedHash != other.Hash() {
return false // can't be same if hash is different
}
// Must compare the actual array elements and not just the array address
// TODO: The hash hashes in all the return states anyway, to we maybe don't need to compare them here?
return slices.Equal(a.returnStates, other.returnStates) &&
slices.EqualFunc(a.parents, other.parents, func(x, y PredictionContext) bool {
return x.Equals(y)
})
}
//// Equals is the default comparison function for ArrayPredictionContext when no specialized
//// implementation is needed for a collection
//func (a *ArrayPredictionContext) Equals(o Collectable[*PredictionContext]) bool {
// if a == o {
// return true
// }
// other, ok := o.(*ArrayPredictionContext)
// if !ok {
// return false
// }
// if a.cachedHash != other.Hash() {
// return false // can't be same if hash is different
// }
//
// // Must compare the actual array elements and not just the array address
// // TODO: The hash hashes in all the return states anyway, to we maybe don't need to compare them here?
// return slices.Equal(a.returnStates, other.returnStates) &&
// slices.EqualFunc(a.parents, other.parents, func(x, y *PredictionContext) bool {
// return x.Equals(y)
// })
//}

// Hash is the default hash function for ArrayPredictionContext when no specialized
// implementation is needed for a collection
func (a *ArrayPredictionContext) Hash() int {
return a.BasePredictionContext.cachedHash
}

func (a *ArrayPredictionContext) String() string {
if a.isEmpty() {
return "[]"
}
s := "["
for i := 0; i < len(a.returnStates); i++ {
if i > 0 {
s = s + ", "
}
if a.returnStates[i] == BasePredictionContextEmptyReturnState {
s = s + "$"
continue
}
s = s + strconv.Itoa(a.returnStates[i])
if a.parents[i] != nil {
s = s + " " + a.parents[i].String()
} else {
s = s + "nil"
}
}
return s + "]"
}
//func (a *ArrayPredictionContext) String() string {
// if a.isEmpty() {
// return "[]"
// }
//
// s := "["
// for i := 0; i < len(a.returnStates); i++ {
// if i > 0 {
// s = s + ", "
// }
// if a.returnStates[i] == BasePredictionContextEmptyReturnState {
// s = s + "$"
// continue
// }
// s = s + strconv.Itoa(a.returnStates[i])
// if a.parents[i] != nil {
// s = s + " " + a.parents[i].String()
// } else {
// s = s + "nil"
// }
// }
//
// return s + "]"
//}
28 changes: 14 additions & 14 deletions runtime/Go/antlr/v4/atn_config.go
Original file line number Diff line number Diff line change
@@ -29,9 +29,9 @@ type ATNConfig interface {
GetSemanticContext() SemanticContext

// GetContext returns the rule invocation stack associated with this configuration
GetContext() PredictionContext
GetContext() *PredictionContext
// SetContext sets the rule invocation stack associated with this configuration
SetContext(PredictionContext)
SetContext(*PredictionContext)

// GetReachesIntoOuterContext returns the count of references to an outer context from this configuration
GetReachesIntoOuterContext() int
@@ -52,7 +52,7 @@ type BaseATNConfig struct {
precedenceFilterSuppressed bool
state ATNState
alt int
context PredictionContext
context *PredictionContext
semanticContext SemanticContext
reachesIntoOuterContext int
}
@@ -69,12 +69,12 @@ func NewBaseATNConfig7(old *BaseATNConfig) ATNConfig { // TODO: Dup - maybe dele
}

// NewBaseATNConfig6 creates a new BaseATNConfig instance given a state, alt and context only
func NewBaseATNConfig6(state ATNState, alt int, context PredictionContext) *BaseATNConfig {
func NewBaseATNConfig6(state ATNState, alt int, context *PredictionContext) *BaseATNConfig {
return NewBaseATNConfig5(state, alt, context, SemanticContextNone)
}

// NewBaseATNConfig5 creates a new BaseATNConfig instance given a state, alt, context and semantic context
func NewBaseATNConfig5(state ATNState, alt int, context PredictionContext, semanticContext SemanticContext) *BaseATNConfig {
func NewBaseATNConfig5(state ATNState, alt int, context *PredictionContext, semanticContext SemanticContext) *BaseATNConfig {
if semanticContext == nil {
panic("semanticContext cannot be nil") // TODO: Necessary?
}
@@ -98,13 +98,13 @@ func NewBaseATNConfig2(c ATNConfig, semanticContext SemanticContext) *BaseATNCon
}

// NewBaseATNConfig1 creates a new BaseATNConfig instance given an existing config, a state, and a context only
func NewBaseATNConfig1(c ATNConfig, state ATNState, context PredictionContext) *BaseATNConfig {
func NewBaseATNConfig1(c ATNConfig, state ATNState, context *PredictionContext) *BaseATNConfig {
return NewBaseATNConfig(c, state, context, c.GetSemanticContext())
}

// NewBaseATNConfig creates a new BaseATNConfig instance given an existing config, a state, a context and a semantic context, other 'constructors'
// are just wrappers around this one.
func NewBaseATNConfig(c ATNConfig, state ATNState, context PredictionContext, semanticContext SemanticContext) *BaseATNConfig {
func NewBaseATNConfig(c ATNConfig, state ATNState, context *PredictionContext, semanticContext SemanticContext) *BaseATNConfig {
if semanticContext == nil {
panic("semanticContext cannot be nil") // TODO: Remove this - probably put here for some bug that is now fixed
}
@@ -115,7 +115,7 @@ func NewBaseATNConfig(c ATNConfig, state ATNState, context PredictionContext, se
return b
}

func (b *BaseATNConfig) InitBaseATNConfig(c ATNConfig, state ATNState, alt int, context PredictionContext, semanticContext SemanticContext) {
func (b *BaseATNConfig) InitBaseATNConfig(c ATNConfig, state ATNState, alt int, context *PredictionContext, semanticContext SemanticContext) {

b.state = state
b.alt = alt
@@ -144,12 +144,12 @@ func (b *BaseATNConfig) GetAlt() int {
}

// SetContext sets the rule invocation stack associated with this configuration
func (b *BaseATNConfig) SetContext(v PredictionContext) {
func (b *BaseATNConfig) SetContext(v *PredictionContext) {
b.context = v
}

// GetContext returns the rule invocation stack associated with this configuration
func (b *BaseATNConfig) GetContext() PredictionContext {
func (b *BaseATNConfig) GetContext() *PredictionContext {
return b.context
}

@@ -248,7 +248,7 @@ type LexerATNConfig struct {
passedThroughNonGreedyDecision bool
}

func NewLexerATNConfig6(state ATNState, alt int, context PredictionContext) *LexerATNConfig {
func NewLexerATNConfig6(state ATNState, alt int, context *PredictionContext) *LexerATNConfig {

return &LexerATNConfig{
BaseATNConfig: BaseATNConfig{
@@ -260,7 +260,7 @@ func NewLexerATNConfig6(state ATNState, alt int, context PredictionContext) *Lex
}
}

func NewLexerATNConfig5(state ATNState, alt int, context PredictionContext, lexerActionExecutor *LexerActionExecutor) *LexerATNConfig {
func NewLexerATNConfig5(state ATNState, alt int, context *PredictionContext, lexerActionExecutor *LexerActionExecutor) *LexerATNConfig {
return &LexerATNConfig{
BaseATNConfig: BaseATNConfig{
state: state,
@@ -291,7 +291,7 @@ func NewLexerATNConfig3(c *LexerATNConfig, state ATNState, lexerActionExecutor *
return lac
}

func NewLexerATNConfig2(c *LexerATNConfig, state ATNState, context PredictionContext) *LexerATNConfig {
func NewLexerATNConfig2(c *LexerATNConfig, state ATNState, context *PredictionContext) *LexerATNConfig {
lac := &LexerATNConfig{
lexerActionExecutor: c.lexerActionExecutor,
passedThroughNonGreedyDecision: checkNonGreedyDecision(c, state),
@@ -301,7 +301,7 @@ func NewLexerATNConfig2(c *LexerATNConfig, state ATNState, context PredictionCon
}

//goland:noinspection GoUnusedExportedFunction
func NewLexerATNConfig1(state ATNState, alt int, context PredictionContext) *LexerATNConfig {
func NewLexerATNConfig1(state ATNState, alt int, context *PredictionContext) *LexerATNConfig {
lac := &LexerATNConfig{
BaseATNConfig: BaseATNConfig{
state: state,
5 changes: 3 additions & 2 deletions runtime/Go/antlr/v4/atn_simulator.go
Original file line number Diff line number Diff line change
@@ -18,12 +18,13 @@ type BaseATNSimulator struct {
decisionToDFA []*DFA
}

func (b *BaseATNSimulator) getCachedContext(context PredictionContext) PredictionContext {
func (b *BaseATNSimulator) getCachedContext(context *PredictionContext) *PredictionContext {
if b.sharedContextCache == nil {
return context
}

visited := NewJStore[PredictionContext, Comparator[PredictionContext]](pContextEqInst)
// TODO: Should this be guarded by a mutex?
visited := NewJStore[*PredictionContext, Comparator[*PredictionContext]](pContextEqInst)

return getCachedBasePredictionContext(context, b.sharedContextCache, visited)
}
6 changes: 3 additions & 3 deletions runtime/Go/antlr/v4/atn_state.go
Original file line number Diff line number Diff line change
@@ -78,9 +78,9 @@ type BaseATNState struct {
transitions []Transition
}

//func NewBaseATNState() *BaseATNState {
// return &BaseATNState{stateNumber: ATNStateInvalidStateNumber, stateType: ATNStateInvalidType}
//}
func NewBaseATNState() *BaseATNState {
return &BaseATNState{stateNumber: ATNStateInvalidStateNumber, stateType: ATNStateInvalidType}
}

func (as *BaseATNState) GetRuleIndex() int {
return as.ruleIndex
6 changes: 3 additions & 3 deletions runtime/Go/antlr/v4/base_prediction_context.go
Original file line number Diff line number Diff line change
@@ -16,9 +16,9 @@ func (b *BasePredictionContext) Equals(_ Collectable[PredictionContext]) bool {
return false
}

func (b *BasePredictionContext) GetParent(i int) PredictionContext {
return nil
}
//func (b *BasePredictionContext) GetParent(i int) PredictionContext {
// return nil
//}

func (b *BasePredictionContext) getReturnState(i int) int {
return 0
2 changes: 1 addition & 1 deletion runtime/Go/antlr/v4/comparators.go
Original file line number Diff line number Diff line change
@@ -29,7 +29,7 @@ var (
dfaStateEqInst = &ObjEqComparator[*DFAState]{}
semctxEqInst = &ObjEqComparator[SemanticContext]{}
atnAltCfgEqInst = &ATNAltConfigComparator[ATNConfig]{}
pContextEqInst = &ObjEqComparator[PredictionContext]{}
pContextEqInst = &ObjEqComparator[*PredictionContext]{}
)

// Equals2 delegates to the Equals() method of type T
19 changes: 4 additions & 15 deletions runtime/Go/antlr/v4/empty_prediction_context.go
Original file line number Diff line number Diff line change
@@ -15,18 +15,7 @@ type EmptyPredictionContext struct {
BaseSingletonPredictionContext
}

func NewEmptyPredictionContext() *EmptyPredictionContext {
return &EmptyPredictionContext{
BaseSingletonPredictionContext: BaseSingletonPredictionContext{
BasePredictionContext: BasePredictionContext{
cachedHash: calculateEmptyHash(),
pcType: PredictionContextEmpty,
},
parentCtx: nil,
returnState: BasePredictionContextEmptyReturnState,
},
}
}

func (e *EmptyPredictionContext) length() int {
return 1
}
@@ -35,9 +24,9 @@ func (e *EmptyPredictionContext) isEmpty() bool {
return true
}

func (e *EmptyPredictionContext) GetParent(_ int) PredictionContext {
return nil
}
//func (e *EmptyPredictionContext) GetParent(_ int) PredictionContext {
// return nil
//}

func (e *EmptyPredictionContext) getReturnState(_ int) int {
return e.returnState
Loading