Split TypeScript types up in codegen

[lunaris]

This PR refactors NodeJS/TypeScript code generation so that input, output and enumeration types are now generated in separate files near their resources. Previously, all inputs, outputs and enumerations would be grouped into a coarse-grained types/* module hierarchy, which hurts type checking performance in the case of large providers such as AWS (see #11558 and #10442, for instance). This commit changes code generation so that we now emit input.ts/output.ts/enums.ts files next to resources and functions as appropriate, rather than bundling them all into one large module. For backwards compatibility, the original module hierarchy has been preserved using re-exports until such a time as we decide to bump the major versions of our provider SDKs to remove it.

I've tested this manually against the AWS Classic Provider, which as well as having many resources also has a series of overlays, and everything appears to build without any changes. Moreover, using the new fine-grained modules in the AWS case results in a sizeable reduction to the number of modules type checked (assuming some other changes to the aforementioned overlays are made), so I think if this works as expected it would pave the way to fix some of the linked performance issues. To give a taste of the difference:

Before

$ git show HEAD:sdk/nodejs/types/input.ts
// *** WARNING: this file was generated by the Pulumi Terraform Bridge (tfgen) Tool. ***
// *** Do not edit by hand unless you're certain you know what you are doing! ***

import * as pulumi from "@pulumi/pulumi";
import * as inputs from "../types/input";
import * as outputs from "../types/output";
import * as enums from "../types/enums";

import * as utilities from "../utilities";

import {RoutingRule} from "../s3";

export interface GetAvailabilityZoneFilter {
    ...
}

...

export namespace accessanalyzer {
  ...
}

(all input types reside in this one module; importing types brings them all in, as does importing e.g. s3/bucket, since that imports types)

After

$ cat sdk/nodejs/types/input.ts
// *** WARNING: this file was generated by the Pulumi Terraform Bridge (tfgen) Tool. ***
// *** Do not edit by hand unless you're certain you know what you are doing! ***

export * from "../input";
export * as accessanalyzer from "../accessanalyzer/input";
export * as acm from "../acm/input";
...

(types re-exported from fine-grained modules for backwards compatibility)

$ cat sdk/nodejs/acm/input.ts

// *** WARNING: this file was generated by the Pulumi Terraform Bridge (tfgen) Tool. ***
// *** Do not edit by hand unless you're certain you know what you are doing! ***

import * as pulumi from "@pulumi/pulumi";
import * as inputs from "./input";
import * as outputs from "./output";

export interface CertificateDomainValidationOption {
    /**
     * Fully qualified domain name (FQDN) in the certificate.
     */
    domainName?: pulumi.Input<string>;
    /**
     * The name of the DNS record to create to validate the certificate
     */
    resourceRecordName?: pulumi.Input<string>;
    /**
     * The type of DNS record to create
     */
    resourceRecordType?: pulumi.Input<string>;
    /**
     * The value the DNS record needs to have
     */
    resourceRecordValue?: pulumi.Input<string>;
}

...

(fine-grained modules that if imported will only pull in minimal dependencies)

Obviously this could be a terrible way of going about it/a terrible idea in general. All feedback welcome 🙏

[pulumi-bot]

Changelog

[uncommitted] (2024-05-14)

Features

• [sdkgen/nodejs] Generate fine-grained types modules when generating TypeScript SDKs
  #16193

[iwahbe]

If we are interested in removing this call after a major version bump, we need to add a mechanism to disable this call somehow.