-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AliECS core Kafka producer #520
Merged
Merged
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
f963751
[core] Refactor serverutil in preparation for eventstream
teo 25fdadd
[core] Events protofile
teo ca1c759
[core] Additional events
teo 0f92acb
[core] Update events.proto+o2control.proto with NewEnvironmentAsync
teo cf40d75
[common] Use events.proto in o2control.proto
teo 8c5f41f
[coconut] Fix Protobuf generator call
teo 34b7a0d
[core] Kafka wrapper
teo 0a7c61b
[core] Emit environment events
teo e983b2c
[coconut] Add asynchronous mode (-y) to coconut env create command
teo fabece2
[coconut] Implement coconut env create -y
teo d13d12b
[core] Add task traits and CallEvent to events.proto
teo 231d3d9
[common] Add CallEvent
teo aa4ac3f
[core] Emit call events to inform on plugin calls
teo 95c41fe
[core] Send EnvId with TaskEvents
teo 225a5f9
[core] Rename busEvent in task.go
teo 48499fb
[core] Add IntegratedServiceEvent and rename Envid field
teo fe2dc90
[core] Push env vars on workflow load
teo 7530576
[common] Allow event creation with specific timestamp
teo 28c3e68
[common] Various additions to events in events.proto
teo 6b7c8fd
[build] Bump dependencies
teo 92d513f
[core] Include parent role path in task events
teo e82a40e
[core] Improve call information in CallEvents
teo dc190db
[core] Emit IntegratedServiceEvents from DCS
teo 8c57a45
[core] Make sure we always output ECS detector codes, not DCS ones
teo 274e73f
[core] Don't forget to include error in DCS ERROR events
teo b3e3461
[core] Better DCS event descriptions
teo 27eec71
[core] Emit ddscheduler events
teo 68b2c46
[core] Remove legacy ODC handlers
teo 16ce849
[core] Emit ODC events
teo db62a9f
[core] Emit TRG events
teo ae22193
[common] Enable AllowAutoTopicCreation in Kafka client
teo 831178d
[core] Correct Kafka topic
teo 9c62431
[build] Generate fdset file for decoding Kafka messages with pq
teo d17a4a1
[core] Emit call events to aliecs.call topic and include envId
teo 8695b2a
[core] Enable IntegratedServiceEvents
teo 31eb36d
[core] Pass IntegratedServiceEvents by ref
teo e733190
[core] Write to Kafka asynchronously
teo 6ffe6b6
[core] Nullify odc Devices list before emitting events
teo d8785f0
[core] Trim down ODC events some more
teo e3a392f
[core] Publish ODC partition state changes
teo e3d59c1
[core] Document events.proto and change currentRunNumber field
teo 62dcce8
[core] Document currently unused topics
teo f7c2786
[docs] Document Kafka producer functionality
teo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next
Next commit
[core] Refactor serverutil in preparation for eventstream
- Loading branch information
commit f96375104baac3cbe287257f254309e9fc32e6d1
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
/* | ||
* === This file is part of ALICE O² === | ||
* | ||
* Copyright 2020-2022 CERN and copyright holders of ALICE O². | ||
* Author: Miltiadis Alexis <miltiadis.alexis@cern.ch> | ||
* Teo Mrnjavac <teo.mrnjavac@cern.ch> | ||
* | ||
* This program is free software: you can redistribute it and/or modify | ||
* it under the terms of the GNU General Public License as published by | ||
* the Free Software Foundation, either version 3 of the License, or | ||
* (at your option) any later version. | ||
* | ||
* This program is distributed in the hope that it will be useful, | ||
* but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
* GNU General Public License for more details. | ||
* | ||
* You should have received a copy of the GNU General Public License | ||
* along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
* | ||
* In applying this license CERN does not waive the privileges and | ||
* immunities granted to it by virtue of its status as an | ||
* Intergovernmental Organization or submit itself to any jurisdiction. | ||
*/ | ||
|
||
package core | ||
|
||
import ( | ||
"sync" | ||
|
||
pb "github.com/AliceO2Group/Control/core/protos" | ||
) | ||
|
||
// SafeStreamsMap is a safe map where the key is usually a | ||
// subscriptionID received from the grpc call and as a value | ||
// a channel where get events from the environment | ||
// and we stream them to the grpc client. | ||
type SafeStreamsMap struct { | ||
mu sync.RWMutex | ||
streams map[string]chan *pb.Event | ||
} | ||
|
||
func (s *SafeStreamsMap) add(id string, ch chan *pb.Event) { | ||
s.mu.Lock() | ||
s.streams[id] = ch | ||
s.mu.Unlock() | ||
} | ||
|
||
func (s *SafeStreamsMap) delete(id string) { | ||
s.mu.Lock() | ||
delete(s.streams, id) | ||
s.mu.Unlock() | ||
} | ||
|
||
func (s *SafeStreamsMap) GetChannel(id string) (ch chan *pb.Event, ok bool) { | ||
s.mu.RLock() | ||
defer s.mu.RUnlock() | ||
ch, ok = s.streams[id] | ||
return | ||
} | ||
|
||
func newSafeStreamsMap() SafeStreamsMap { | ||
return SafeStreamsMap{ | ||
streams: make(map[string]chan *pb.Event), | ||
} | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understood correctly, you are just moving this somewhere else... is it connected to kafka producer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, streamutil.go. It's part of a general cleanup at the very start of this effort.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok