Plugin framework

The existing plugin framework allows adopters to publish npm or PyPI packages that declare custom fields for CommonGrants schema objects. We want to expand this framework to also support bidirectional data transformations — toCommon/fromCommon functions that convert between a source system’s native data shape and the CommonGrants schema. We also want to organize this framework to support other potential future features with minimal rework.

Plugin authors are free to implement toCommon/fromCommon as plain hand-written functions. The SDK also provides a buildTransforms() / build_transforms() utility (see ADR-0017) that can generate these functions from separate declarative mapping objects, but using it is not required.

How should the Plugin object be structured to support both custom field declarations and bidirectional transforms, while enabling clean dependency injection and remaining stable as the protocol’s object list grows?

Questions

• Should the top-level Plugin structure group by feature (meta, client, schemas, extensions) or by object (Opportunity, Application, …)?
• Should client configuration (auth, transport, rate-limiting) sit alongside per-object schemas, or be lifted to the top level as a system-level concern?
• Should custom fields and transforms be coupled in the same package, or allowed independently?

Decision drivers

• The framework must be implementable in both the Python and TypeScript SDKs with as consistent an interface as possible.
• The extensions config must be serializable and able to pass validation (JSON-safe), and must be combinable across multiple extension packages via mergeExtensions() (TypeScript) / merge_extensions() (Python).
• Existing plugin packages that declare only custom fields should remain valid with minimal changes.
• The SDK interface should support clean dependency injection — it must be possible to pass client or schemas as a coherent unit without reassembling them from per-object branches.
• Auth, transport, and rate-limiting are system-level concerns that belong to a single client, not distributed across per-object branches.
• The top-level Plugin surface should be short and stable — adding new protocol objects should not expand the top-level key set.

Decision

We decided to:

1. Keep “plugin” as the unified term for both the published npm/PyPI packages in the website catalog and the runtime SDK object. No change to PluginSourceEntry or src/content/plugins/index.json. The existing definePlugin() function is expanded to accept the full set of top-level fields described below.

2. Use functional grouping at the top level with four keys — meta, client, schemas, and extensions — rather than grouping by object name at the root. Custom filters (filters) are an authoring-time input on DefinePluginOptions / define_plugin() and surface through the Client returned by getClient(), so they don’t add a fifth top-level key on the runtime Plugin shape.

3. Use per-object grouping inside schemas where it reflects real coupling: each object’s native schema, CommonGrants schema, and bidirectional transforms are tightly coupled and change together.

4. Expand definePlugin() to accept all top-level fields — meta, client, schemas, and extensions — rather than only extensions. extensions will contain any parts of the plugin that can be merged with other plugins by using mergeExtensions(), which has a flag for handling key conflicts.

5. Make all top-level Plugin fields optional so adopters can publish a plugin that provides only the features they need — for example, custom fields only — and expand to include transforms, client config, or additional schemas incrementally over time.

6. Plugin authors provide toCommon / fromCommon as functions; mappings are one way to generate them. The SDK exposes buildTransforms() / build_transforms() as a public utility wrapping the existing mapping runtimes. PluginExtensions.schemas.<Object> gains an optional mappings key carrying JSON-safe toCommon / fromCommon mapping objects; when those are declared and no explicit transform is supplied in schemas.<Object>, definePlugin() invokes buildTransforms() automatically. Mappings for each direction are author-provided — buildTransforms() does not invert one direction into the other, because many-to-one handlers like switch are not reversible.

7. toCommon / fromCommon return a TransformResult<T> of { result, errors } unconditionally; mapping definitions are validated at buildTransforms() call time. Partial failure is routine for cross-schema transforms — field handlers can emit warnings that do not invalidate a record — so the transform surface is safe by default rather than throwing. Runtime schema validation (Zod .parse() / Pydantic model_validate()) surfaces as entries in errors rather than thrown exceptions. In the current PoC, this validation is opt-in at the buildTransforms() call site via the commonModel / common_model parameter — when supplied, validation runs inside toCommon against the fully extended generated schema. In the full SDK, definePlugin() will additionally inject validation when auto-generating transforms from extensions.schemas.<Object>.mappings. Plugin authors using hand-written transforms are responsible for their own validation. Consumers apply their own rule for what counts as success — strict adopters treat any non-empty errors as failure, lenient adopters tolerate warnings. Mappings passed to buildTransforms() are checked at the call site, failing fast on structural errors, unknown handlers, or unresolvable field paths.

8. Custom handlers are registered per utility call, not globally. buildTransforms() accepts an optional handlers argument for registering additional handler names. Per-call scoping keeps behavior explicit and testable; name collisions with the default set raise at buildTransforms() call time rather than silently shadowing them. Handler-name lookup must not resolve inherited attributes, because mapping JSON can be reconstituted from untrusted sources via mergeExtensions().

9. Transformation errors carry structured context. SDK-emitted transformation errors extend a single PluginError base carrying field path, handler name, source value, and underlying cause, enabling programmatic reasoning without parsing error text. The source value may contain PII when transforming applicant data; adopters are responsible for redacting it before logging or re-raising, and the SDK does not redact by default.

The resulting Plugin shape:

plugin.meta        // name, version, sourceSystem, capabilities
plugin.getClient   // (config: ClientConfig) => Client; memoized by definePlugin()
plugin.extensions  // serializable; used by mergeExtensions()
plugin.schemas.<ObjectName> = { native, common, toCommon, fromCommon }

Example interface

TypeScript

type PluginCapability =
  | "customFields" // declares custom fields on CommonGrants schema objects
  | "customFilters" // declares custom filter parameters for resource methods
  | "transforms" // provides toCommon/fromCommon transformation functions
  | "client"; // provides a runtime client (auth, transport, resource methods)

interface PluginMeta {
  name: string;
  version?: string; // optional; if omitted, definePlugin() infers it from the package's package.json
  sourceSystem: string;
  capabilities?: PluginCapability[];
}

// Defined in lib/ts-sdk/src/extensions/types.ts — reproduced here for reference
interface CustomFieldSpec {
  name?: string; // optional; dict key is used as the display name fallback
  fieldType: CustomFieldType; // enum defined in the SDK
  value?: z.ZodTypeAny; // optional Zod schema to validate the value; defaults based on fieldType
  description?: string;
}

// Unconditional return shape for toCommon / fromCommon (see Decision #7).
// `result` is the transformed value; `errors` aggregates PluginErrors emitted during
// transformation and runtime schema validation. `errors` may be empty on full success,
// or non-empty alongside a result when handlers emit non-fatal warnings.
interface TransformResult<T> {
  result: T;
  errors: PluginError[];
}

// Runtime type — produced by definePlugin(), not provided directly by plugin authors
interface ObjectSchemas<TNative, TCommon> {
  native: ZodType<TNative>;
  common: ZodType<TCommon>;
  toCommon: (native: TNative) => TransformResult<TCommon>;
  fromCommon: (common: TCommon) => TransformResult<TNative>;
}

// Input type — provided by plugin authors inside DefinePluginOptions.schemas
// common is intentionally absent: the plugin config file cannot import from generated/
// since it is the input to generation. definePlugin() injects common during compilation
// from ObjectSchemasInput → ObjectSchemas, resolved from the generated model classes.
interface ObjectSchemasInput<TNative = unknown, TCommon = unknown> {
  native?: ZodType<TNative>; // defaults to Record<string, unknown> if omitted
  toCommon?: (native: TNative) => TransformResult<TCommon>;
  fromCommon?: (common: TCommon) => TransformResult<TNative>;
}

// Scalar types only — filters are query parameters, not schema fields
type CustomFilterType = "string" | "number" | "integer" | "boolean";

interface CustomFilterSpec {
  filterType: CustomFilterType;
  description?: string;
}

// Per-object config shape inside extensions.schemas — mirrors Python's PluginExtensionsSchema
interface PluginExtensionsObjectConfig {
  customFields?: Record<string, CustomFieldSpec>;
  // Optional declarative mappings in ADR-0017 format. When present and no explicit
  // toCommon / fromCommon is supplied in schemas.<Object>, definePlugin() auto-invokes
  // buildTransforms() on these. Each direction is author-provided; see Decision #6.
  mappings?: {
    toCommon?: Record<string, unknown>; // ADR-0017 mapping: native → CommonGrants
    fromCommon?: Record<string, unknown>; // ADR-0017 mapping: CommonGrants → native
  };
}

// Serializable portion of the plugin config — safe to store as JSON.
// schemas keys are restricted to ExtensibleSchemaName (the known set of CommonGrants
// objects that support custom fields), following the existing SchemaExtensions pattern.
interface PluginExtensions {
  meta?: Partial<PluginMeta>;
  schemas?: Partial<Record<ExtensibleSchemaName, PluginExtensionsObjectConfig>>;
}

// ClientConfig is defined by the plugin author to declare the system-specific inputs
// they require (e.g. auth token, base URL, max page size, timeouts).
interface ClientConfig {
  [key: string]: unknown;
}

// Client is a placeholder for the SDK's runtime client type (not shown here).
// No `filters` key on Plugin — handled by the Client returned by getClient() (see Decision #2).
interface Plugin {
  meta?: PluginMeta;
  getClient?: (config: ClientConfig) => Client;
  extensions?: PluginExtensions; // serializable
  schemas?: Partial<
    Record<ExtensibleSchemaName, ObjectSchemas<unknown, unknown>>
  >;
}

// Input object for definePlugin(). Using a named-options object makes it easy to add
// new inputs over time without breaking existing callers.
interface DefinePluginOptions {
  meta?: PluginMeta;
  // Plugin authors provide a factory function; definePlugin() wraps it with memoization
  // so the same Client instance is returned for equivalent configs automatically.
  getClient?: (config: ClientConfig & { auth?: AuthMethod }) => Client;
  extensions?: PluginExtensions; // serializable
  // Plugin authors provide input schemas and transforms; definePlugin() compiles them
  // into the full ObjectSchemas runtime type, merging any customFields from extensions.
  schemas?: Partial<Record<ExtensibleSchemaName, ObjectSchemasInput>>;
  filters?: Partial<
    Record<ExtensibleSchemaName, Record<string, CustomFilterSpec>>
  >;
}

// Factory: all options are optional so adopters can start with only what they need
// and expand incrementally.
//
// definePlugin compiles DefinePluginOptions into a Plugin by:
//   - extending the base CommonGrants schema with any declared customFields → common
//   - native defaults to Record<string, unknown> if omitted (extensions is JSON-safe;
//     runtime Zod schemas cannot be included)
//   - wrapping getClient with memoization so the same Client instance is returned
//     for equivalent configs automatically
//
// toCommon / fromCommon may be plain hand-written functions, generated via
// buildTransforms() and passed in schemas, or auto-generated by definePlugin()
// itself — when extensions.schemas.<Object>.mappings is declared and schemas.<Object>
// provides no explicit transform, definePlugin() invokes buildTransforms() internally.
// All transforms return TransformResult<T>; definePlugin() validates the result field
// at runtime with schema.parse / model_validate and appends any validation failures
// to the errors array rather than throwing (see Decision #7).
function definePlugin(options: DefinePluginOptions): Plugin;

// Combine multiple extension objects (e.g. from separate packages).
// Exact signature shape (overloads, param structure) is provisional pending SDK pin in #744.
function mergeExtensions(
  extensions: PluginExtensions[],
  options?: { onConflict?: "error" | "firstWins" | "lastWins" }, // defaults to "error"
): PluginExtensions;

// Handler signature matches ADR-0017 runtime conventions.
type Handler = (value: unknown, context: unknown) => unknown;

// Utility: generates toCommon and fromCommon functions from separate declarative
// mapping objects (ADR-0017 format). Using this utility is optional — plugin authors
// may provide plain hand-written functions instead. Mappings are validated at call
// time (see Decision #7); the optional `handlers` argument registers custom handler
// names for this call only (see Decision #8).
// When commonModel is provided, toCommon calls commonModel.parse (Zod) on its output
// and appends any validation errors to TransformResult.errors rather than throwing.
// commonModel must be the fully extended generated schema (e.g. the generated
// Opportunity with typed customFields), not the base schema — passing a base schema
// silently weakens validation of typed custom fields.
// The underlying mapping runtime normalizes model/schema instances to plain objects
// at the entry point, so fromCommon can receive the validated output of toCommon
// and field paths still resolve correctly.
function buildTransforms<TNative, TCommon>(
  toCommonMapping: Record<string, unknown>, // ADR-0017 mapping from native → CommonGrants
  fromCommonMapping: Record<string, unknown>, // ADR-0017 mapping from CommonGrants → native
  handlers?: Record<string, Handler>,
  commonModel?: ZodType<TCommon>, // must be the generated extended schema, not the base
): {
  toCommon: (native: TNative) => TransformResult<TCommon>;
  fromCommon: (common: TCommon) => TransformResult<TNative>;
};

// Base class for SDK-emitted transformation errors (see Decision #9).
interface PluginError extends Error {
  path?: string;
  handler?: string;
  sourceValue?: unknown;
  cause?: unknown;
}

Python

from dataclasses import dataclass
from typing import Any, Callable, Generic, Literal, TypeVar
from pydantic import BaseModel, ConfigDict, Field
# Client is a placeholder for the SDK's runtime client type (not shown here)

TNative = TypeVar('TNative')
TCommon = TypeVar('TCommon')
T = TypeVar('T')

# Unconditional return shape for to_common / from_common (see Decision #7).
# `result` is the transformed value; `errors` aggregates PluginErrors emitted during
# transformation and runtime schema validation. `errors` may be empty on full success,
# or non-empty alongside a result when handlers emit non-fatal warnings.
@dataclass
class TransformResult(Generic[T]):
    result: T
    errors: list['PluginError']

# Defined in lib/python-sdk/common_grants_sdk/extensions/specs.py — reproduced here for reference
@dataclass
class CustomFieldSpec:
    """Custom Field spec class to support adding custom fields"""
    field_type: CustomFieldType  # enum defined in the SDK
    value: Any | None = None     # optional; used to validate the field value
    name: str = ""               # optional; dict key is used as the display name fallback
    description: str = ""

# Runtime type — produced by define_plugin(), not provided directly by plugin authors
@dataclass
class ObjectSchemas(Generic[TNative, TCommon]):
    native: type[TNative]   # expects a Pydantic BaseModel subclass
    common: type[TCommon]   # expects a Pydantic BaseModel subclass
    to_common: Callable[[TNative], TransformResult[TCommon]]
    from_common: Callable[[TCommon], TransformResult[TNative]]

# Input type — provided by plugin authors inside define_plugin(schemas=...)
# common is intentionally absent: cg_config.py cannot import from generated/ since
# it is the input to generation. define_plugin() injects common during compilation
# from ObjectSchemasInput → ObjectSchemas, resolved from the generated model classes.
@dataclass
class ObjectSchemasInput(Generic[TNative, TCommon]):
    native: type[TNative] | None = None  # defaults to dict[str, Any] if omitted
    to_common: Callable[[TNative], TransformResult[TCommon]] | None = None
    from_common: Callable[[TCommon], TransformResult[TNative]] | None = None

# Scalar types only — filters are query parameters, not schema fields
CustomFilterType = Literal['string', 'number', 'integer', 'boolean']

@dataclass
class CustomFilterSpec:
    filter_type: CustomFilterType
    description: str = ""

PluginCapability = Literal['customFields', 'customFilters', 'transforms', 'client']

class PluginMeta(BaseModel):
    model_config = ConfigDict(populate_by_name=True)

    name: str
    version: str | None = None  # optional; if omitted, define_plugin() infers it from the package's pyproject.toml / importlib.metadata
    source_system: str = Field(alias='sourceSystem')
    capabilities: list[PluginCapability] | None = None

# Equivalent to TypeScript's Partial<PluginMeta>. Defined as a separate model
# rather than reusing PluginMeta because Pydantic does not have a built-in Partial.
# Note: if PluginMeta gains new required fields, this class must be updated manually.
# Drift can be caught with: assert PluginMeta.model_fields.keys() == PluginExtensionsMeta.model_fields.keys()
class PluginExtensionsMeta(BaseModel):
    model_config = ConfigDict(populate_by_name=True)

    name: str | None = None
    version: str | None = None
    source_system: str | None = Field(default=None, alias='sourceSystem')
    capabilities: list[PluginCapability] | None = None

class ObjectMappings(BaseModel):
    model_config = ConfigDict(populate_by_name=True)

    # ADR-0017 mappings. Each direction is author-provided; see Decision #6.
    to_common: dict[str, Any] | None = Field(default=None, alias='toCommon')      # native → CommonGrants
    from_common: dict[str, Any] | None = Field(default=None, alias='fromCommon')  # CommonGrants → native

class PluginExtensionsSchema(BaseModel):
    model_config = ConfigDict(populate_by_name=True)

    custom_fields: dict[str, CustomFieldSpec] | None = Field(default=None, alias='customFields')
    # Optional declarative mappings. When present and no explicit to_common / from_common
    # is supplied in schemas[obj], define_plugin() auto-invokes build_transforms() on these.
    mappings: ObjectMappings | None = None

class PluginExtensions(BaseModel):
    meta: PluginExtensionsMeta | None = None
    schemas: dict[str, PluginExtensionsSchema] | None = None

ClientConfig = dict[str, Any]  # plugin authors define their own keys (auth, base_url, timeout, etc.)

# No `filters` field on Plugin — handled by the Client returned by get_client() (see Decision #2).
@dataclass
class Plugin:
    meta: PluginMeta | None = None
    get_client: Callable[[ClientConfig], Client] | None = None
    extensions: PluginExtensions | None = None
    schemas: dict[ExtensibleSchemaName, ObjectSchemas[Any, Any]] | None = None

# All params are optional — adopters can start with only what they need and expand
# incrementally. Unlike TypeScript, Python supports named optional params at the
# function root, so no DefinePluginOptions wrapper object is needed.
#
# define_plugin compiles inputs into a Plugin by:
#   - extending the base CommonGrants model with any declared custom_fields → common
#   - native defaults to dict[str, Any] if omitted (extensions is JSON-safe; runtime
#     Pydantic models cannot be included)
#   - wrapping get_client with memoization so the same Client instance is returned
#     for equivalent configs automatically
#
# to_common / from_common may be plain hand-written callables, generated via
# build_transforms() and passed in schemas, or auto-generated by define_plugin()
# itself — when extensions.schemas[obj].mappings is declared and schemas[obj] provides
# no explicit transform, define_plugin() invokes build_transforms() internally.
# All transforms return TransformResult[T]; define_plugin() validates the result field
# at runtime with model_validate and appends any validation failures to the errors
# list rather than raising (see Decision #7).
def define_plugin(
    meta: PluginMeta | None = None,
    get_client: Callable[[ClientConfig], Client] | None = None,
    extensions: PluginExtensions | None = None,
    schemas: dict[ExtensibleSchemaName, ObjectSchemasInput[Any, Any]] | None = None,
    filters: dict[ExtensibleSchemaName, dict[str, CustomFilterSpec]] | None = None,
) -> Plugin: ...

# Exact signature shape is provisional pending SDK pin in #744.
def merge_extensions(
    *extensions: PluginExtensions,
    on_conflict: Literal["error", "firstWins", "lastWins"] = "error",
) -> PluginExtensions: ...

# Handler signature matches ADR-0017 runtime conventions.
Handler = Callable[[Any, Any], Any]

# Utility: generates to_common and from_common callables from separate declarative
# mapping dicts (ADR-0017 format). Using this utility is optional — plugin authors
# may provide plain hand-written callables instead. Mappings are validated at call
# time (see Decision #7); the optional `handlers` argument registers custom handler
# names for this call only (see Decision #8).
# When common_model is provided, to_common calls model_validate on its output and
# appends any ValidationErrors to TransformResult.errors rather than raising.
# common_model must be the fully extended generated model class (e.g.
# generated/schemas.py's Opportunity), not the base class — passing a base class
# silently weakens validation of typed custom fields.
# transform_from_mapping normalizes Pydantic model instances to plain dicts via
# model_dump(mode="json") at the entry point, so from_common can receive the
# validated model output of to_common and field paths still resolve correctly.
def build_transforms(
    to_common_mapping: dict[str, Any],   # ADR-0017 mapping from native → CommonGrants
    from_common_mapping: dict[str, Any], # ADR-0017 mapping from CommonGrants → native
    handlers: dict[str, Handler] | None = None,
    common_model: type[BaseModel] | None = None,  # must be the generated extended model, not the base
) -> tuple[
    Callable[[Any], TransformResult[Any]],
    Callable[[Any], TransformResult[Any]],
]: ...

# Base class for SDK-emitted transformation errors (see Decision #9).
class PluginError(Exception):
    path: str | None
    handler: str | None
    source_value: Any
    cause: BaseException | None

Example author usage

TypeScript

// buildTransforms() is a utility that generates toCommon/fromCommon from declarative
// mappings (ADR-0017). Using it is optional — plain functions work just as well.
const { toCommon, fromCommon } = buildTransforms(
  // toCommon: native grants.gov shape → CommonGrants Opportunity
  {
    title: "data.opportunity_title",
    status: {
      value: {
        match: {
          field: "data.opportunity_status",
          case: { posted: "open", archived: "closed" },
          default: "custom",
        },
      },
    },
  },
  // fromCommon: CommonGrants Opportunity → native grants.gov shape
  {
    "data.opportunity_title": "title",
    "data.opportunity_status": {
      value: {
        match: {
          field: "status",
          case: { open: "posted", closed: "archived" },
          default: "custom",
        },
      },
    },
  },
);

const plugin = definePlugin({
  meta: {
    name: "grants-gov-plugin",
    version: "1.0.0",
    sourceSystem: "grants.gov",
  },
  // definePlugin memoizes getClient — the same Client is returned for equivalent configs.
  getClient: (config: ClientConfig & { auth?: AuthMethod }) =>
    new Client({
      baseUrl: config.baseUrl ?? "https://api.grants.gov",
      timeout: config.timeout,
      pageSize: config.pageSize,
      maxItems: config.maxItems,
      auth: config.auth,
    }),
  extensions: {
    schemas: {
      Opportunity: {
        customFields: {
          programArea: {
            fieldType: CustomFieldType.String,
            description: "HHS program area code",
          },
          legacyGrantId: {
            fieldType: CustomFieldType.Integer,
            description: "Numeric ID from the legacy grants system",
          },
        },
      },
    },
  },
  schemas: {
    Opportunity: {
      native: GrantsGovOpportunitySchema,
      toCommon,
      fromCommon,
    },
  },
});

// Combine extensions from multiple packages before constructing the plugin
const merged = mergeExtensions([baseExtensions, grantsGovExtensions]);
const mergedPlugin = definePlugin({ extensions: merged });

// Calling getClient() with a config object — memoized, so repeated calls return the same instance
const client = plugin.getClient({ auth: Auth.bearer("token"), pageSize: 50 });

Python

# build_transforms() is a utility that generates to_common/from_common from declarative
# mappings (ADR-0017). Using it is optional — plain callables work just as well.
to_common, from_common = build_transforms(
    # to_common: native grants.gov shape → CommonGrants Opportunity
    to_common_mapping={
        'title': 'data.opportunity_title',
        'status': {
            'value': {
                'match': {
                    'field': 'data.opportunity_status',
                    'case': {'posted': 'open', 'archived': 'closed'},
                    'default': 'custom',
                },
            },
        },
    },
    # from_common: CommonGrants Opportunity → native grants.gov shape
    from_common_mapping={
        'data.opportunity_title': 'title',
        'data.opportunity_status': {
            'value': {
                'match': {
                    'field': 'status',
                    'case': {'open': 'posted', 'closed': 'archived'},
                    'default': 'custom',
                },
            },
        },
    },
)

plugin = define_plugin(
    meta=PluginMeta(name='grants-gov-plugin', version='1.0.0', source_system='grants.gov'),  # source_system serializes as 'sourceSystem' in JSON
    # define_plugin memoizes get_client — the same Client is returned for equivalent configs.
    get_client=lambda config: Client(config=Config(
        base_url=config.get('base_url', 'https://api.grants.gov'),
        api_key=config['api_key'],
        timeout=config.get('timeout', 10.0),
        page_size=config.get('page_size', 100),
        list_items_limit=config.get('list_items_limit', 1000),
    )),
    extensions=PluginExtensions(
        schemas={
            'Opportunity': PluginExtensionsSchema(
                custom_fields={
                    'programArea': CustomFieldSpec(field_type=CustomFieldType.STRING, description='HHS program area code'),
                    'legacyGrantId': CustomFieldSpec(field_type=CustomFieldType.INTEGER, description='Numeric ID from legacy system'),
                },
            ),
        },
    ),
    schemas={
        'Opportunity': ObjectSchemasInput(
            native=GrantsGovOpportunity,
            to_common=to_common,
            from_common=from_common,
        ),
    },
)


# Combine extensions from multiple packages before constructing the plugin
merged = merge_extensions(base_extensions, grants_gov_extensions)
merged_plugin = define_plugin(extensions=merged)

# Calling get_client() with a config dict — memoized, so repeated calls return the same instance
client = plugin.get_client({'api_key': 'abc123', 'page_size': 50})

Example consumer usage

TypeScript

import { grantsGovPlugin } from "grants-gov-plugin";

// Get a configured client for this source system
const client = grantsGovPlugin.getClient({
  auth: Auth.bearer(process.env.GRANTS_GOV_API_KEY),
  pageSize: 25,
});

// Use the compiled schemas to transform native data into CommonGrants shape.
// toCommon / fromCommon return TransformResult<T> = { result, errors } — consumers
// apply their own strict-vs-lenient rule for what counts as success.
const { toCommon } = grantsGovPlugin.schemas.Opportunity;
const { result, errors } = toCommon(rawGrantsGovData);
if (errors.length === 0) {
  use(result); // strict: treat any error (including handler warnings) as failure
} else {
  // lenient: use result despite warnings; inspect errors for context
  for (const err of errors) console.warn(err.path, err.message);
}

// Batch transformation is a plain .map — each element carries its own result and errors
const items = rawBatch.map(toCommon);
const successful = items
  .filter((r) => r.errors.length === 0)
  .map((r) => r.result);

// Inspect what the plugin declares about itself
console.log(grantsGovPlugin.meta.sourceSystem); // "grants.gov"
console.log(grantsGovPlugin.meta.capabilities); // ["customFields", "transforms", "client"]

Python

from grants_gov_plugin import grants_gov_plugin

# Get a configured client for this source system
client = grants_gov_plugin.get_client({
    'api_key': os.environ['GRANTS_GOV_API_KEY'],
    'page_size': 25,
})

# Use the compiled schemas to transform native data into CommonGrants shape.
# to_common / from_common return TransformResult[T] = {result, errors} — consumers
# apply their own strict-vs-lenient rule for what counts as success.
to_common = grants_gov_plugin.schemas['Opportunity'].to_common
outcome = to_common(raw_grants_gov_data)
if not outcome.errors:
    use(outcome.result)  # strict: treat any error (including handler warnings) as failure
else:
    # lenient: use outcome.result despite warnings; inspect errors for context
    for err in outcome.errors:
        print(err.path, err)

# Batch transformation is a plain list comprehension — each element carries its own result and errors
items = [to_common(x) for x in raw_batch]
successful = [r.result for r in items if not r.errors]

# Inspect what the plugin declares about itself
print(grants_gov_plugin.meta.source_system)   # "grants.gov"
print(grants_gov_plugin.meta.capabilities)    # ["customFields", "transforms", "client"]

Consequences

• Positive consequences
  • Client stays singular — getClient() is memoized by definePlugin(), so one source system always produces one Client instance regardless of how many times getClient() is called
  • Top-level surface (meta, client, schemas, extensions) is short, closed, and stable — adding protocol objects adds a key under schemas only
  • Dependency injection works along functional lines: pass getClient, pass Schemas, pass Extensions as coherent units without needing to reassemble from per-object branches
  • mergeExtensions() / merge_extensions() operates on flat, serializable data at the root, not on deeply nested per-object branches
  • Per-object grouping inside schemas preserves the real coupling between native schema, CommonGrants schema, and bidirectional transforms — they share type signatures and change together
  • Mirrors the SDK module structure (client, schemas, extensions), so Plugin reads as a system-specific version of the existing SDK rather than a different mental model
  • toCommon/fromCommon can be plain hand-written functions, generated via buildTransforms() and passed in schemas, or auto-generated by definePlugin() from mappings declared in extensions — plugin authors are not required to use a declarative mapping format
  • buildTransforms() accepts separate toCommonMapping and fromCommonMapping objects, reflecting that the two directions of a bidirectional transform are distinct
  • toCommon / fromCommon return TransformResult<T> so partial failure surfaces as data, batch processing is a plain .map, and consumers apply their own strict-vs-lenient rule for what counts as success; structured PluginError lets adopters reason about those failures programmatically without parsing error text
  • Custom handlers are registered per-call on buildTransforms(), not globally — behavior stays explicit and testable, and collisions with the default set raise at buildTransforms() call time
  • customFields is optional — the customFields-only config structure remains valid; existing plugin packages require only minimal code changes to adopt definePlugin()
  • All top-level Plugin fields are optional — adopters can start with only what they need and expand incrementally
• Negative consequences
  • extensions (serializable config) and plugin (runtime object including client) are distinct concepts that adopters must understand separately

Criteria

• Backward compatible: Existing custom-fields-only plugins remain valid without changes
• SDK-friendly: Config shape maps naturally to Pydantic/Zod one-model-at-a-time usage inside schemas
• Language-agnostic config: The extensions JSON document uses camelCase keys (customFields, fieldType, sourceSystem) in both SDKs — Python source uses snake_case attributes with camelCase alias fields, matching the existing SDK convention
• Clear naming: A single term — “plugin” — is used consistently across the registry catalog and SDK
• Supports both capabilities: Custom field declarations and bidirectional transforms can coexist or be used independently; transforms may be hand-written or generated from declarative mappings
• Incremental adoption: All top-level fields are optional, so adopters can start with only what they need
• DI-friendly: Functional top-level keys can be passed as coherent units without reassembly
• Stable surface: New protocol objects do not expand the top-level key set

Options considered

• Object-first structure with adapted model/schema (no separate Plugin class)
• Pure object-first structure with “Plugin” for both registry and SDK
• Functional top-level with per-object schema grouping (chosen)

Evaluation

Side-by-side

Criteria	Object-first / Adapted Schema	Pure object-first / Plugin	Functional top-level / per-object schemas
Backward compatible	✅	✅	✅
SDK-friendly	✅	✅	✅
Language-agnostic config	✅	✅	✅
Clear naming	🟡	✅	✅
Supports both capabilities	✅	✅	✅
DI-friendly	🟡	🔴	✅
Stable surface	🟡	🔴	✅

Option 1: Object-first structure, adapted model/schema (no separate Plugin class)

Instead of constructing a separate Plugin object, the SDK returns an extended version of the model/schema itself with the transform baked in. Adopters call native parse/validate methods directly on the returned object.

TypeScript

// TypeScript: createPlugin returns an extended Zod schema (ZodEffects), not a Plugin object
const opportunityPlugin = createPlugin(opportunitySchema, pluginConfig);
const opportunity = opportunityPlugin.parse(grantsGovData); // native Zod
const result = opportunityPlugin.safeParse(grantsGovData); // native Zod non-throwing

Python

# Python: create_plugin returns a new Pydantic model class with a custom validator applied
OpportunityPlugin = create_plugin(Opportunity, plugin_config)
opportunity = OpportunityPlugin.model_validate(grants_gov_data)  # native Pydantic

Bottom line

Object-first / Adapted Schema is best if:

• we want the SDK return type to feel like a first-class schema object, not a wrapper
• and we’re willing to accept that the “Plugin” concept is implicit rather than a named class

• Pros
  • Very idiomatic — .parse() / safeParse() in Zod and .model_validate() in Pydantic are the expected call sites
  • No new runtime class name to explain; the adapted schema is still recognizably a schema
  • Backward compatible — both keys optional, custom_fields-only is valid
• Cons
  • No named Plugin type to import, document, or type-hint against
  • Client, auth, and transport have no natural home in this model
  • DI requires passing each model’s plugin separately rather than a unified Schemas object — callers must accept one plugin per object rather than a single Schemas unit (DI-friendly: 🟡)
  • Top-level surface tracks the object list indirectly via function calls (createPlugin(opportunitySchema, ...), createPlugin(applicationSchema, ...)), but there is no stable type that enumerates supported objects (Stable surface: 🟡)
  • In Python, create_plugin must dynamically generate a new model class, which is less transparent

Option 2: Pure object-first structure, “Plugin” for both registry and SDK

Config and runtime object both keyed by CommonGrants model name at the root. meta and client sit alongside object keys but are not themselves objects.

TypeScript

interface Plugin {
  meta?: PluginMeta;
  client?: Client;
  Opportunity?: ObjectPluginConfig;
  Application?: ObjectPluginConfig;
  // ... one key per protocol object
}

Python

# Python equivalent — same object-keyed shape
@dataclass
class Plugin:
    meta: PluginMeta | None = None
    client: Client | None = None
    Opportunity: ObjectPluginConfig | None = None
    Application: ObjectPluginConfig | None = None
    # ... one field per protocol object

Bottom line

Pure object-first / Plugin is best if:

• we want authoring a new object’s support to feel self-contained during development
• but we’re willing to accept a top-level key set that grows with the protocol and scattered client config

• Pros
  • Co-location: all of an object’s config is in one branch during authoring
  • Single unified term — Plugin is used for both the registry catalog and the SDK runtime object
• Cons
  • Top-level keys track the protocol’s object list, which is long and open-ended — surface grows as protocol grows and raises questions about which of 100+ schemas belongs at the top level
  • Client, auth, and transport are system-level but must either be duplicated per object or kept implicit alongside object keys, creating an awkward mix of concerns
  • Filters attach to resource methods rather than schemas, and resource methods aren’t consistent across objects (e.g. opportunities.list/get/search vs applications.start/submit), creating a poor fit
  • DI requires reassembling a flat view across all object branches (e.g. an allSchemas helper) — working against the grain of the structure
  • mergeExtensions() must deeply merge nested per-object branches rather than operating on a flat serializable root

Option 3: Functional top-level with per-object schema grouping (chosen)

Top-level keys are functional (meta, client, schemas, extensions). Per-object grouping is used only inside schemas, where it reflects real coupling between native schemas, CommonGrants schemas, and bidirectional transforms.

interface Plugin {
  meta?: PluginMeta;
  getClient?: (config: ClientConfig) => Client;
  extensions?: PluginExtensions;
  schemas?: Partial<Record<ExtensibleSchemaName, ObjectSchemas>>; // per-object grouping only here
}

Bottom line

Functional top-level / per-object schemas is best if:

• we want a stable, short top-level surface that doesn’t grow with the protocol’s object list
• and we want a singular client and clean DI while still co-locating the tightly-coupled schema/transform pairs per object

• Pros
  • Short, stable top-level surface — meta, client, schemas, extensions tracks a closed list regardless of how many protocol objects exist
  • Client is singular — getClient() is memoized by definePlugin(), so one source system always produces one Client instance
  • DI works along functional lines: pass getClient, pass Schemas, pass Extensions as units
  • mergeExtensions() operates on flat, serializable data at the root, not deeply nested per-object branches
  • Per-object grouping inside schemas preserves real coupling — native schema, CommonGrants schema, toCommon, and fromCommon share type signatures and change together
  • Mirrors the SDK module structure (client, schemas, extensions) — Plugin is a system-specific version of the existing SDK, not a different mental model
• Cons
  • extensions (serializable config) and plugin (runtime object including client) are distinct concepts that adopters must learn separately