Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter connectors better and make connector search page safer #699

Merged
merged 11 commits into from
Feb 24, 2025
Merged

Filter connectors better and make connector search page safer #699

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
cc2be2d
When searching this logo was for sure missing at time
travjenkins Feb 21, 2025
6a27129
Needed to import the graphql from gatsby
travjenkins Feb 21, 2025
846abaa
adding back in missing imports
travjenkins Feb 21, 2025
9cb8e67
adding imports
travjenkins Feb 21, 2025
0767e8e
We handle missing tags in the normalize call so we don't have to filter
travjenkins Feb 21, 2025
3d53bc8
Marking anything that _could_ be undefined as optional
travjenkins Feb 21, 2025
987b1ae
cleaning up logging
travjenkins Feb 21, 2025
ed2e00d
removing testing logs
travjenkins Feb 21, 2025
396d323
Updating caniuse
travjenkins Feb 21, 2025
33ff29c
Upgrading to the latest strapi plugin
travjenkins Feb 21, 2025
b228685
Merge branch 'master' into travjenkins/bug/fix-broken-connector-rende…
travjenkins Feb 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 71 additions & 57 deletions src/components/Connectors/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import ChevronRight from '@mui/icons-material/ChevronRight';
import { graphql, Link, useStaticQuery } from 'gatsby';
import { graphql, useStaticQuery, Link } from 'gatsby';
import { GatsbyImage, StaticImage } from 'gatsby-plugin-image';
import { useMemo, useState } from 'react';
import { useLunr } from 'react-lunr';
Expand Down Expand Up @@ -65,48 +65,66 @@ const ConnectorCard = ({
slug,
type,
showType = false,
}: ReturnType<typeof normalizeConnector> & { showType?: boolean }) => (
<Link to={`${slug}`}>
<div className={connectorCard}>
<div className={connectorCardTop}>
<GatsbyImage
image={logo?.childImageSharp?.gatsbyImageData}
alt={`${title} Logo`}
className={clsx(connectorPostCardImage, 'icon-wrapper')}
loading="eager"
/>
{recommended || showType ? (
<div style={{ flexGrow: 1 }} />
) : null}
{recommended ? (
<div>
<p className={connectorPostCardRecommended}>
RECOMMENDED
</p>
</div>
) : null}
{showType ? (
<>
{recommended ? <div style={{ flexBasis: 4 }} /> : null}
}: ReturnType<typeof normalizeConnector> & { showType?: boolean }) => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This didn't change much but looks like it did because I added a explicit return.

return (
<Link to={`${slug}`}>
<div className={connectorCard}>
<div className={connectorCardTop}>
{!logo?.childImageSharp?.gatsbyImageData ? (
<div
className={clsx(
connectorPostCardImage,
'icon-wrapper'
)}
/>
) : (
Comment on lines +73 to +80
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logo was 100% missing while running a quick search. So making this safer. Eventually we might want to add a placeholder or something while this is loading.

<GatsbyImage
image={logo.childImageSharp.gatsbyImageData}
alt={`${title} Logo`}
className={clsx(
connectorPostCardImage,
'icon-wrapper'
)}
loading="eager"
/>
)}
{recommended || showType ? (
<div style={{ flexGrow: 1 }} />
) : null}
{recommended ? (
<div>
<p className={connectorPostCardRecommended}>
{type === 'capture' ? 'SOURCE' : 'DESTINATION'}
RECOMMENDED
</p>
</div>
</>
) : null}
{showType ? (
<>
{recommended ? (
<div style={{ flexBasis: 4 }} />
) : null}
<div>
<p className={connectorPostCardRecommended}>
{type === 'capture'
? 'SOURCE'
: 'DESTINATION'}
</p>
</div>
</>
) : null}
</div>
<h4>{title}</h4>
{shortDescription?.length > 0 ? (
<p>{truncate(shortDescription || '', 100)}</p>
) : null}
<div style={{ flexGrow: 1 }} />
<span className={connectorCardReadMore}>
Read More <ChevronRight />
</span>
</div>
<h4>{title}</h4>
{shortDescription?.length > 0 ? (
<p>{truncate(shortDescription || '', 100)}</p>
) : null}
<div style={{ flexGrow: 1 }} />
<span className={connectorCardReadMore}>
Read More <ChevronRight />
</span>
</div>
</Link>
);
</Link>
);
};

export const Connectors = ({
connectorType,
Expand Down Expand Up @@ -158,10 +176,6 @@ export const Connectors = ({
const mappedConnectors = useMemo(
() =>
postgres.allConnectors.nodes
.filter(
(connector) =>
connector?.connectorTagsByConnectorIdList?.length > 0
)
.map(normalizeConnector)
.filter(
(connector) =>
Expand Down Expand Up @@ -193,14 +207,12 @@ export const Connectors = ({
[postgres]
);

const logosByConnectorId = useMemo(
() =>
Object.assign(
{},
...mappedConnectors.map((con) => ({ [con.id]: con.logo }))
),
[mappedConnectors]
);
const logosByConnectorId = useMemo(() => {
return Object.assign(
{},
...mappedConnectors.map((con) => ({ [con.id]: con.logo }))
);
}, [mappedConnectors]);

const [query, setQuery] = useState('');
const results = useLunr(
Expand Down Expand Up @@ -240,14 +252,16 @@ export const Connectors = ({
</div>
<div className={connectorCards}>
{(query.length > 0 ? results : mappedConnectors).map(
(connector) => (
<ConnectorCard
key={connector.id}
{...connector}
logo={logosByConnectorId[connector.id]}
showType={showAllConnectors}
/>
)
(connector) => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another explicit return

return (
<ConnectorCard
key={connector.id}
{...connector}
logo={logosByConnectorId[connector.id]}
showType={showAllConnectors}
/>
);
}
)}
</div>
</div>
Expand Down
4 changes: 0 additions & 4 deletions src/components/ConnectorsLink/index.tsx
View file
Open in desktop
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now check this in normalizeConnector.

Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,6 @@ const ConnectorsLink = ({
const [captureConnectors, materializationConnectors] = useMemo(() => {
const mapped: ReturnType<typeof normalizeConnector>[] = connectors
.filter((connector) => connector.logo !== null)
.filter(
(connector) =>
connector?.connectorTagsByConnectorIdList?.length > 0
)
.map(normalizeConnector)
.filter((connector) => connector !== undefined);

Expand Down
24 changes: 12 additions & 12 deletions src/templates/connection.tsx
View file
Open in desktop
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should never happen - but being safe

Original file line number Diff line number Diff line change
Expand Up @@ -44,25 +44,25 @@ const Connector = ({
<Layout>
<Hero
sourceConnector={{
title: source_mapped.title,
logo: source_mapped.logo,
title: source_mapped?.title,
logo: source_mapped?.logo,
}}
destConnector={{
title: dest_mapped.title,
logo: dest_mapped.logo,
title: dest_mapped?.title,
logo: dest_mapped?.logo,
}}
/>
<FromConnector
title={source_mapped.title}
logo={source_mapped.logo}
longDescription={source_mapped.longDescription}
shortDescription={source_mapped.shortDescription}
title={source_mapped?.title}
logo={source_mapped?.logo}
longDescription={source_mapped?.longDescription}
shortDescription={source_mapped?.shortDescription}
/>
<ToConnector
title={dest_mapped.title}
logo={dest_mapped.logo}
longDescription={dest_mapped.longDescription}
shortDescription={dest_mapped.shortDescription}
title={dest_mapped?.title}
logo={dest_mapped?.logo}
longDescription={dest_mapped?.longDescription}
shortDescription={dest_mapped?.shortDescription}
/>
<Testimonials />
<EstuaryFlowVideo />
Expand Down
4 changes: 2 additions & 2 deletions src/templates/connector/Hero/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ import {
type HeroProps = {
connector: {
title: string;
logo: ImageDataLike;
logo: ImageDataLike | null | undefined;
type: ConnectorType;
};
};

const Hero = ({ connector: { title, logo, type } }: HeroProps) => {
const logoImage = getImage(logo);
const logoImage = logo ? getImage(logo) : null;

return (
<LightSwoopingLinesRightDirectionBackground>
Expand Down
4 changes: 2 additions & 2 deletions src/templates/connector/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ const Connector = ({
href: '/integrations',
},
{
title: mappedConnector.title,
title: mappedConnector?.title,
},
]}
/>
Expand Down Expand Up @@ -101,7 +101,7 @@ export const Head = ({
case 'capture':
return [
`Move ${mappedConnector?.title} to Any Destination, Real-time ETL & CDC`,
`Effortlessly move ${mappedConnector.title} data to any destination in real-time or batch with Estuary's no-code ETL & CDC platform. Free and easy to use. Get started now.`,
`Effortlessly move ${mappedConnector?.title} data to any destination in real-time or batch with Estuary's no-code ETL & CDC platform. Free and easy to use. Get started now.`,
];
default:
return [
Expand Down
6 changes: 5 additions & 1 deletion src/utils.ts
View file
Open in desktop
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add all the filter into the normalizer so that things like search results do not get fed wrong connectors

Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,12 @@ import {
const CONNECTOR_IMAGE_RE = /(source|materialize)-([a-z0-9\-]+)/;

export const normalizeConnector = (connector: any) => {
if (connector.imageName.includes('ghcr.io/estuary/dekaf')) {
if (
// Exclude any Dekaf connector
connector.imageName.includes('ghcr.io/estuary/dekaf') ||
// Exclude connectors without a tag (Kelkoo)
connector?.connectorTagsByConnectorIdList?.length < 1
) {
return undefined;
}

Expand Down