Skip to content
This repository has been archived by the owner on Feb 26, 2024. It is now read-only.

Devise new experimental @truffle/from-hardhat package for compatibility translation #5420

Merged
merged 5 commits into from
Aug 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/from-hardhat/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": ["../../.eslintrc.package.json"]
}
2 changes: 2 additions & 0 deletions packages/from-hardhat/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
dist
node_modules
8 changes: 8 additions & 0 deletions packages/from-hardhat/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# @truffle/from-hardhat

> :warning: **This package is experimental and FOR INTERNAL USE ONLY.**

This package translates Hardhat project information into Truffle's own formats.

For information on using this package (until we get it more ready for public
use), please see [`./src/api.ts`](src/api.ts).
47 changes: 47 additions & 0 deletions packages/from-hardhat/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"name": "@truffle/from-hardhat",
"description": "Import Hardhat project information into Truffle-native formats",
"license": "MIT",
"author": "g. nicholas d'andrea <gnidan@trufflesuite.com>",
"homepage": "https://github.com/trufflesuite/truffle/tree/master/packages/from-hardhat#readme",
"repository": {
"type": "git",
"url": "https://github.com/trufflesuite/truffle.git",
"directory": "packages/from-hardhat"
},
"bugs": {
"url": "https://github.com/trufflesuite/truffle/issues"
},
"version": "0.1.0-0",
"main": "dist/src/index.js",
"scripts": {
"build": "ttsc",
"prepare": "yarn build",
"test": "exit 0"
},
"dependencies": {
"@truffle/compile-common": "^0.7.32",
"@truffle/compile-solidity": "^6.0.38",
"@truffle/config": "^1.3.34",
"debug": "^4.3.1",
"find-up": "^2.1.0",
"semver": "^5.7.1"
},
"devDependencies": {
"@types/find-up": "^2.1.0",
"@types/node": "^18.6.5",
"hardhat": "^2.10.1",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cds-amal @fainashalts FYI I realized that we can benefit from Hardhat's own types by keeping this as a devDependency safely... if we just don't actually export anything with these types in the signature. This is the approach I'm taking here.

Just letting you know because this breaks the assumptions of our earlier conversation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this compare to making it an optional peerDepenency?

Copy link
Contributor Author

@gnidan gnidan Aug 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this compare to making it an optional peerDepenency?

Good question! Firstly, it's worth noting that package.json's peerDependenciesMeta field (which is how you mark peerDeps as optional) was added in NPM v7, and we need to support v6 still. But this is a minor quibble.

The main concern here, IMO, is the question "what does Truffle's Node.js-land need from the "hardhat" NPM package?"

  • Types means we need at least a devDep
  • We need more than devDep if we want to have a shared Node.js runtime between Truffle and Hardhat... as in, if we want to use Hardhat as a library

I looked at using HH as a library, but, like Truffle, it's really not intended for use via straight require() (or import). I got it working in #5410... so this approach is viable albeit awkward. But think about what we're saying when Truffle does require("hardhat")... like, "here you go, Node! have another, different one of me!". Even without the risk of cross-clobbering each other's globals / other runtime modification shenanigans, the memory footprint and whatnot... oy!

So my conclusion was basically... screw peerDependencies; HH is prescriptive: you use it via npx hardhat. Truffle can use HH via npx hardhat. Keep the playgrounds separate :)

Copy link
Contributor Author

@gnidan gnidan Aug 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth following up that there is a counter-argument that we could just use peerDependencies for the $(npm bin) or whatever. But still... I think there's value in treating HH as an executable... like, you're not going to try to shove Docker into your package.json - you just shell out to docker and error nicely if it's not there.

Oh, and another follow-up argument might be: we could use peerDependencies to leverage npm/yarn's semver powers to avoid manual version compatibility checks ourselves at runtime... and yeah, we could, but I don't think it's worth the added surface area of complexity around managing this peerDependency... like, who gets the actual dependency? Truffle? Like, oof.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah that makes sense, and feel free to ignore my repeat question in the review below.

I was just thinking that it'd be very nice if there was some way to import just the types from the hardhat package without needing to add all of their transitive dependencies into our node_modules tree and our yarn.lock (even if it is just for dev purposes).

If our usage of their types is minimal, perhaps it'd be worth extracting only the types that we need into a local type declaration? Though I could also easily argue against that approach due to the potential effort involved in doing that, along with the increased maintenance burden that will be required.

Sure would be nice if there was a code-gen utility for that sort of thing, though!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I see your concern. Not sure copy+pasting the types is acceptable? Sounds like follow-on work? Or maybe we could get HH to publish types separately?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think if we make changes to this approach, it's definitely something that's a future concern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I resolved this, but realized that neither of the two people who you pinged at the start have replied, so I've unresolved it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise please consider my part in this resolved.

"ttypescript": "1.5.13",
"typescript": "^4.3.5",
"typescript-transform-paths": "3.3.1"
},
"keywords": [
"ethereum",
"solidity",
"truffle",
"hardhat"
],
"publishConfig": {
"access": "public"
}
}
164 changes: 164 additions & 0 deletions packages/from-hardhat/src/api.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
import { promises as fs } from "fs";
import semver from "semver";

import type * as Hardhat from "hardhat/types";
import type * as Common from "@truffle/compile-common";
import type TruffleConfig from "@truffle/config";

import {
supportedHardhatVersionRange,
supportedHardhatBuildInfoFormats
} from "./constants";
import * as Compilation from "./compilation";
import * as Config from "./config";

import {
checkHardhat,
askHardhatConsole,
askHardhatVersion
} from "./ask-hardhat";
import { EnvironmentOptions } from "./options";

/**
* Checks for the existence of a Hardhat project configuration and asserts
* that the local installed version of Hardhat matches this package's
* supported version range.
*
* @param options to control process environment (e.g. working directory)
* @return Promise<void> when expectation holds
* @throws NotHardhatError when not in a Hardhat project directory
* @throws IncompatibleHardhatError if Hardhat has unsupported version
*/
export const expectHardhat = async (
options?: EnvironmentOptions
): Promise<void> => {
const isHardhat = await checkHardhat(options);

if (!isHardhat) {
throw new NotHardhatError();
}

const hardhatVersion = await askHardhatVersion(options);

if (!semver.satisfies(hardhatVersion, supportedHardhatVersionRange)) {
throw new IncompatibleHardhatVersionError(hardhatVersion);
}
};

/**
* Thrown when no Hardhat project is found
*/
export class NotHardhatError extends Error {
constructor() {
super("Current working directory is not part of a Hardhat project");
}
}

/**
* Thrown when Hardhat was detected but with an incompatible version
*/
export class IncompatibleHardhatVersionError extends Error {
constructor(detectedVersion: string) {
super(
`Expected Hardhat version compatible with ${supportedHardhatVersionRange}, got: ${detectedVersion}`
);
}
}

/**
* Constructs a @truffle/config object based on the Hardhat config.
*
* WARNING: except for fields documented here, the values present on the
* returned @truffle/config object MUST be regarded as unsafe to use.
*
* The returned `config` is defined to contain the following:
*
* - `config.networks` with configurations for all Hardhat-configured
* networks, provided:
* - The configured network is not the built-in `hardhat` network
* - The configured network defines a `url` property
*
* Note: this function ignores all properties other than `url`,
* including any information that can be used for computing
* cryptographic signatures. THIS FUNCTION DOES NOT READ PRIVATE KEYS.
*
* Suffice to say:
*
* THIS FUNCTION'S BEHAVIOR IS EXPERIMENTAL AND SHOULD ONLY BE USED IN
* SPECIFICALLY KNOWN-SUPPORTED USE CASES (like reading for configured
* network urls)
*
* @param options to control process environment (e.g. working directory)
* @return Promise<TruffleConfig>
*
* @dev This function shells out to `npx hardhat console` to ask the Hardhat
* runtime environment for a fully populated config object.
*/
export const prepareConfig = async (
options?: EnvironmentOptions
): Promise<TruffleConfig> => {
const hardhatConfig = (await askHardhatConsole(
`hre.config`,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth noting about this approach: this only works because hre.config is serializable.

I am so grateful and so jealous that they made this serializable :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of the struggle of forging our own path in the early days of the Ethereum ecosystem, I suppose. Also the node ecosystem and what is considered best practice has advanced just a touch since 2015!

Like, I think at the time that the original truffle-config.js was designed executable config was the hotness, as it was (and still is) super flexible. Now it's somewhat frowned upon for multi-modal projects like ours due this exact concern.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's trade-offs... like, I also had the thought about how easy it is to read private keys from memory. Our approach clamps that down, since we only store mnemonics inside closures.

options
)) as Hardhat.HardhatConfig;

return Config.fromHardhatConfig(hardhatConfig);
};

/**
* Constructs an array of @truffle/compile-common `Compilation` objects
* corresponding one-to-one with Hardhat's persisted results of each solc
* compilation.
*
* WARNING: this function only supports Hardhat projects written entirely
* in solc-compatible languages (Solidity, Yul). Behavior of this function
* for Hardhat projects using other languages is undefined.
*
* @param options to control process environment (e.g. working directory)
* @return Promise<Compilation[]> from @truffle/compile-common
*
* @dev This function shells out to `npx hardhat console` to ask the Hardhat
* runtime environment for the location of the project build info
* files
*/
export const prepareCompilations = async (
options?: EnvironmentOptions
): Promise<Common.Compilation[]> => {
const compilations = [];

const buildInfoPaths = (await askHardhatConsole(
`artifacts.getBuildInfoPaths()`,
options
)) as string[];

for (const buildInfoPath of buildInfoPaths) {
const buildInfo: Hardhat.BuildInfo = JSON.parse(
(await fs.readFile(buildInfoPath)).toString()
);

const { _format } = buildInfo;

if (!supportedHardhatBuildInfoFormats.has(_format)) {
throw new IncompatibleHardhatBuildInfoFormatError(_format);
}

const compilation = Compilation.fromBuildInfo(buildInfo);

compilations.push(compilation);
}

return compilations;
};

/**
* Thrown when the build-info format detected has an incompatible version
*/
export class IncompatibleHardhatBuildInfoFormatError extends Error {
constructor(detectedFormat: string) {
super(
`Expected build-info to be one of ["${[
...supportedHardhatBuildInfoFormats
].join('", "')}"], got: "${detectedFormat}"`
);
}
}
107 changes: 107 additions & 0 deletions packages/from-hardhat/src/ask-hardhat.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import { spawn } from "child_process";

import findUp from "find-up";

import { validHardhatConfigFilenames } from "./constants";
import { EnvironmentOptions, withDefaultEnvironmentOptions } from "./options";

/**
* Returns a Promise to a boolean that is true if and only if
* the detected or specified environment is part of a Hardhat project.
*
* (i.e., if the working directory or any of its parents has a Hardhat config)
*/
export const checkHardhat = async (
options?: EnvironmentOptions
): Promise<boolean> => {
const { workingDirectory } = withDefaultEnvironmentOptions(options);

// search recursively up for a hardhat config
const hardhatConfigPath = await findUp(validHardhatConfigFilenames, {
cwd: workingDirectory
});

return !!hardhatConfigPath;
};

/**
* Reads version information via `npx hardhat --version`
*/
export const askHardhatVersion = async (
options?: EnvironmentOptions
): Promise<string> =>
new Promise((accept, reject) => {
const { workingDirectory } = withDefaultEnvironmentOptions(options);

const hardhat = spawn(`npx`, ["hardhat", "--version"], {
stdio: ["pipe", "pipe", "inherit"],
cwd: workingDirectory
});

let output = "";
hardhat.stdout.on("data", data => {
output = `${output}${data}`;
});

hardhat.once("close", code => {
if (code !== 0) {
return reject(new Error(`Hardhat exited with non-zero code ${code}`));
}

return accept(output);
});
});

export interface AskHardhatConsoleOptions {
// turn off json stringify/parse
raw?: boolean;
}

export const askHardhatConsole = async (
expression: string,
{
raw = false,
...options
}: AskHardhatConsoleOptions & EnvironmentOptions = {}
): Promise<string | unknown> =>
new Promise((accept, reject) => {
const { workingDirectory } = withDefaultEnvironmentOptions(options);

const hardhat = spawn(`npx`, ["hardhat", "console"], {
stdio: ["pipe", "pipe", "inherit"],
cwd: workingDirectory
});

// we'll capture the stdout
let output = "";
hardhat.stdout.on("data", data => {
output = `${output}${data}`;
});

// setup close event before writing to stdin because we're sending eof
hardhat.once("close", code => {
if (code !== 0) {
return reject(new Error(`Hardhat exited with non-zero code ${code}`));
}

if (raw) {
return accept(output);
}

try {
return accept(JSON.parse(output));
} catch (error) {
return reject(error);
}
});

hardhat.stdin.write(`
Promise.resolve(${expression})
.then(${
raw
? `console.log`
: `(resolved) => console.log(JSON.stringify(resolved))`
})
`);
hardhat.stdin.end();
});
Loading