Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoIp location #2465

Merged
merged 82 commits into from
May 6, 2024
Merged
Show file tree
Hide file tree
Changes from 78 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
ded3943
cdn-location-without-data-generation
ptesavol Feb 20, 2024
732020b
x# Please enter a commit message to explain why this merge is necessary,
ptesavol Feb 20, 2024
bfd57f0
fix fetch timeouts
ptesavol Feb 20, 2024
0738f6c
move cdn-location higher in package.json, decrease fetch timeout to 2s
ptesavol Feb 20, 2024
3ddbc79
changes according to review comments
ptesavol Feb 20, 2024
b8417f7
more review fixes
ptesavol Feb 21, 2024
d628283
add data geration scripts and documentation
ptesavol Feb 26, 2024
e6595ab
merged with main
ptesavol Feb 26, 2024
4839109
x# Please enter a commit message to explain why this merge is necessary,
ptesavol Mar 13, 2024
a77bf33
add airport coordinates to airportCodeToRegionNumber.ts
ptesavol Mar 13, 2024
02d3c2e
udpate getLocalRegion.ts to handle the new airport data with coordinates
ptesavol Mar 13, 2024
45d555f
add functionality for getting airport code and region by coordinates
ptesavol Mar 13, 2024
d0fcf57
add package-lock
ptesavol Mar 13, 2024
bdf6143
add geoip packages
ptesavol Mar 13, 2024
9eacf32
add latitude and longitude to ConnectivityResponse
ptesavol Mar 13, 2024
fcacc6c
make generatePeerDescriptorCallBack async
ptesavol Mar 13, 2024
37872bc
make DhtNode get local region by coordinates
ptesavol Mar 13, 2024
335c3a0
working version with node-geolite2-redist
ptesavol Mar 22, 2024
2c78b98
own downloader in dht/helpers
ptesavol Mar 27, 2024
8b999b6
move geoip-location to separate package, fix browser tests
ptesavol Apr 2, 2024
fec1622
merge with main
ptesavol Apr 2, 2024
513b9ff
merged with main
ptesavol Apr 2, 2024
7284f01
fix ci
ptesavol Apr 2, 2024
988c61c
rename geoIpDatabasePath to geoIpDatabaseFolder for clarity
ptesavol Apr 9, 2024
1c90133
remove extra references from tsconfig
ptesavol Apr 9, 2024
70e2d83
precompute paths to variables in the beginning of functions, set perm…
ptesavol Apr 9, 2024
6105d3d
update package lock
ptesavol Apr 9, 2024
4a4deef
rename airportCodeToRegionNumber to airportCodeToRegion
ptesavol Apr 9, 2024
668bb44
add getLocalRegionWithCache()
ptesavol Apr 9, 2024
1e4ca73
merge with main
ptesavol Apr 9, 2024
077d905
update README to include airportCodeToRegionNumber -> airportCodeToRe…
ptesavol Apr 9, 2024
8072f8a
regenerated package lock
ptesavol Apr 9, 2024
d222c31
take package-lock from main
ptesavol Apr 10, 2024
9b4cc6c
fix tests in ci
ptesavol Apr 10, 2024
a92a8c9
add geoip-location references to tsconfigs
ptesavol Apr 10, 2024
9894921
trying to fix build
ptesavol Apr 10, 2024
34d42b0
fix build
juslesan Apr 10, 2024
ad8e990
add local caching of db files into tests to make them work on slow in…
ptesavol Apr 15, 2024
0f03283
eslint
ptesavol Apr 15, 2024
42a155b
increase test times in test/unit/downloadGeoIpDatabase.test.ts
ptesavol Apr 15, 2024
1179de8
increase timeouts in test/unit/GeoIpLocator.test.ts
ptesavol Apr 15, 2024
0da42dd
increase timeout on GeoIpLocator-intervals.test.ts
ptesavol Apr 15, 2024
f89ae67
decrease timeout on GeoIpLocator-intervals.test.ts
ptesavol Apr 15, 2024
bd0c0d8
wait dbs to be downloaded in test/unit/GeoIpLocator-intervals.test.ts
ptesavol Apr 15, 2024
1707eac
add caching on filesystem to TestServer
ptesavol Apr 16, 2024
86048ef
merge with main
ptesavol Apr 16, 2024
e22d322
package-lock from main
ptesavol Apr 16, 2024
f2e1b48
version update
ptesavol Apr 16, 2024
54211a1
resolve race condition
ptesavol Apr 17, 2024
2f8e3ff
fix dependencies
ptesavol Apr 17, 2024
159fd8a
concurrency fixes
ptesavol Apr 17, 2024
418b61c
remove too strict memoty test
ptesavol Apr 17, 2024
ade2c52
Take geoip-location server into use in Broker (#2503)
harbu Apr 17, 2024
9dec0e2
respond to review comments
ptesavol Apr 22, 2024
6fb4e18
add debug logging
ptesavol Apr 22, 2024
35afd2a
merge with main
ptesavol Apr 22, 2024
a0872f0
move TestServer in tarHelper.test.ts to use beforeEach/afterEach
ptesavol Apr 23, 2024
a4ca8de
move to using java-style getter getRemoteIpAddress() in ServerWebSock…
ptesavol Apr 23, 2024
510baed
use default cache age in getLocalRegionWithCache()
ptesavol Apr 23, 2024
e57d68c
Merge branch 'main' into geoip-location
ptesavol Apr 23, 2024
2ef315d
change port of GeoIpConnectivityChecking.test.ts
ptesavol Apr 23, 2024
ca9ebcd
add more time for express to start up
ptesavol Apr 23, 2024
212e35e
merged
ptesavol Apr 23, 2024
cd1310f
remove unnecessary undefined types
juslesan Apr 24, 2024
ef6347a
refactor & reformat
juslesan Apr 24, 2024
d86def0
reduce test tiemouts
juslesan Apr 24, 2024
c686095
move pipeline
juslesan Apr 24, 2024
abc90a6
eslint
juslesan Apr 24, 2024
cb843d9
Merge branch 'main' into geoip-location
juslesan Apr 29, 2024
77e06b0
fix mock typing
juslesan Apr 29, 2024
44068a0
reformat
juslesan Apr 29, 2024
ac846ad
protocol version 1.1
juslesan Apr 29, 2024
daf4e4f
reformat
juslesan Apr 29, 2024
fd2696a
refactor cache
juslesan Apr 29, 2024
9ab34c7
move mock restore
juslesan Apr 29, 2024
608048a
Merge branch 'main' into geoip-location
juslesan Apr 29, 2024
71590a5
Merge branch 'main' into geoip-location
juslesan Apr 29, 2024
934a37a
undefined check for weird getRemoteIpAddress cases
juslesan Apr 29, 2024
281b708
fix test
juslesan Apr 29, 2024
c66988e
falsy
juslesan Apr 29, 2024
1e10d59
Merge branch 'main' into geoip-location
juslesan May 6, 2024
73b789e
Changelog
juslesan May 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ jobs:
uses: ./.github/workflows/test-setup.yml
with:
package: cdn-location
geoip-location:
needs: build
uses: ./.github/workflows/test-setup.yml
with:
package: geoip-location
protocol:
needs: build
uses: ./.github/workflows/test-setup.yml
Expand Down
114 changes: 111 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"packages/utils",
"packages/test-utils",
"packages/cdn-location",
"packages/geoip-location",
"packages/protocol",
"packages/proto-rpc",
"packages/autocertifier-client",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ describe('StreamrChallenger', () => {
},
websocketHost: '127.0.0.1',
websocketPortRange: { min: 12323, max: 12323 },
createLocalPeerDescriptor: () => mockPeerDescriptor1
createLocalPeerDescriptor: async () => mockPeerDescriptor1
}),
metricsContext: new MetricsContext()
})
Expand Down
14 changes: 12 additions & 2 deletions packages/broker/src/broker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,17 @@ export const createBroker = async (configWithoutDefaults: Config): Promise<Broke
const config = validateConfig(configWithoutDefaults, BROKER_CONFIG_SCHEMA)
const plugins: Plugin<any>[] = Object.keys(config.plugins).map((name) => createPlugin(name, config))
applyPluginClientConfigs(plugins, config.client)
const streamrClient = new StreamrClient(config.client)
const streamrClient = new StreamrClient({
...config.client,
network: {
...config.client.network,
controlLayer: {
...config.client.network?.controlLayer,
geoIpDatabaseFolder: config.client.network?.controlLayer?.geoIpDatabaseFolder ??
'~/.streamr/geoipdatabases' // TODO: more cleaner solution?
}
}
})

let httpServer: HttpServer | HttpsServer | undefined

Expand Down Expand Up @@ -70,7 +80,7 @@ export const createBroker = async (configWithoutDefaults: Config): Promise<Broke
}

process.on('uncaughtException', (err) => {
logger.fatal( 'Encountered uncaughtException', { err })
logger.fatal('Encountered uncaughtException', { err })
process.exit(1)
})

Expand Down
2 changes: 1 addition & 1 deletion packages/browser-test-runner/webpack.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ module.exports = function({ entry, libraryName, alias = {} }) {
},
mode: 'development',
entry,
devtool: 'source-map',
devtool: 'eval-source-map',
module: {
rules: [
{
Expand Down
2 changes: 2 additions & 0 deletions packages/cdn-location/.eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ node_modules/**
dist/**
.idea/**
src/proto/**
data-generation/final-data/airportCodeToRegion.ts
src/airportCodeToRegion.ts
Loading
Loading