ESM migration: Add .js extensions for node16 moduleResolution

docs/esm-migration-plan.md

@@ -0,0 +1,262 @@
+# ESM Migration Plan: Add File Extensions to Imports
+
+## Overview
+
+This document outlines the plan to migrate from TypeScript's deprecated `"moduleResolution": "node"` (node10) to `"moduleResolution": "node16"` or `"nodenext"`. This change is necessary because the published ESM packages have extensionless imports that don't work correctly in modern ESM environments.
+
+## Issues Fixed
+
+This migration will resolve the following issues:
+
+- **#154** - Upgrade `moduleResolution` from `node` to `node16` or `nodenext` in tsconfig
+- **#110** - Published ESM code has imports without file extensions
+- **#64** - expressions: ERR_MODULE_NOT_FOUND attempting to run example demo script  
+- **#146** - Can not import `@actions/workflow-parser`
+
+## Problem Statement
+
+### Current State
+
+All packages use `"moduleResolution": "node"`:
+
+| Package | moduleResolution | TypeScript |
+|---------|------------------|------------|
+| expressions | `"node"` | ^4.7.4 |
+| workflow-parser | `"node"` | ^4.8.4 |
+| languageservice | `"node"` | ^4.8.4 |
+| languageserver | `"node"` | ^4.8.4 |
+| browser-playground | `"Node16"` ✅ | ^4.9.4 |
+
+This causes TypeScript to emit code like:
+```javascript
+// Published to npm - INVALID ESM
+export { Expr } from "./ast";  // Missing .js extension!
+```
+
+### Why This Fails
+
+ESM in Node.js 12+ **requires** explicit file extensions. When users try to import these packages:
+
+```javascript
+// User's code
+import { Expr } from "@actions/expressions";
+```
+
+Node.js fails with:
+```
+Error [ERR_MODULE_NOT_FOUND]: Cannot find module '.../node_modules/@actions/expressions/dist/ast'
+```
+
+## Migration Strategy
+
+### Option A: TypeScript 5.7+ with `rewriteRelativeImportExtensions` (Recommended)
+
+TypeScript 5.7 introduced a new compiler option that automatically rewrites `.ts` extensions to `.js` in output:
+
+```jsonc
+{
+  "compilerOptions": {
+    "moduleResolution": "node16",  // or "nodenext"
+    "rewriteRelativeImportExtensions": true
+  }
+}
+```
+
+**Source code:**
+```typescript
+import { Expr } from "./ast.ts";
+```
+
+**Compiled output:**
+```javascript
+export { Expr } from "./ast.js";
+```
+
+**Pros:**
+- Source uses `.ts` extensions (matches actual files)
+- Works with Deno (which requires `.ts` extensions)
+- TypeScript automatically transforms to `.js`
+- Modern, forward-looking approach
+
+**Cons:**
+- Requires TypeScript 5.7+
+- Relatively new feature
+- **BUG:** See "Known Issues" section below
+
+### Option B: Manual `.js` Extensions
+
+Use `.js` extensions in source TypeScript files:
+
+```typescript
+import { Expr } from "./ast.js";  // Points to .ts file, but use .js extension
+```
+
+**Pros:**
+- Works with TypeScript 4.7+ (with node16 moduleResolution)
+- Well-established pattern
+- No post-processing needed
+- Works with ts-jest without extra configuration
+
+**Cons:**
+- Confusing - `.js` files don't exist at write time
+- Doesn't work with Deno out of the box
+
+### Recommendation
+
+**Use Option B** (manual `.js` extensions). Option A with `rewriteRelativeImportExtensions` has compatibility issues with ts-jest and requires additional workarounds.
+
+---
+
+## Known Issues and Workarounds (December 2025)
+
+### 1. TypeScript Version Conflicts in Monorepo
+
+**Problem:** The root `node_modules/typescript` was version 4.9.5 (pulled in by `ts-node` and `tsutils` dependencies), while workspace packages specified `^5.8.3`.
+
+**Symptoms:** 
+- `npx tsc --version` showed 4.9.5
+- `require('typescript').version` in ts-jest showed 5.8.3
+- Confusing build failures
+
+**Solution:** Add npm overrides in root `package.json`:
+```json
+{
+  "overrides": {
+    "typescript": "5.8.3"
+  }
+}
+```
+
+### 2. ts-jest Compatibility with TypeScript 5.9+
+
+**Problem:** ts-jest 29.4.6 uses `typescript.JSDocParsingMode.ParseAll` which doesn't exist in TypeScript's ES module exports.
+
+**Error:**
+```
+TypeError: Cannot read properties of undefined (reading 'ParseAll')
+at Object.<anonymous> (node_modules/ts-jest/dist/compiler/ts-compiler.js:43:123)
+```
+
+**Root Cause:** ts-jest accesses `typescript_1.default.JSDocParsingMode.ParseAll` but TypeScript has no default export in ESM.
+
+**Solution:** 
+- Use ts-jest 29.0.3 (older version that doesn't use this API)
+- OR wait for ts-jest fix
+- **Stay on TypeScript 5.8.3, not 5.9+**
+
+### 3. TypeScript `rewriteRelativeImportExtensions` Bug with .d.ts Files
+
+**Problem:** TypeScript's `rewriteRelativeImportExtensions: true` correctly rewrites `.ts` → `.js` in `.js` output files, but **incorrectly keeps `.ts` extensions in `.d.ts` declaration files**.
+
+**Example:**
+- Source: `export { Expr } from "./ast.ts";`
+- Output `index.js`: `export { Expr } from "./ast.js";` ✅ Correct
+- Output `index.d.ts`: `export { Expr } from "./ast.ts";` ❌ Wrong (should be `.js`)
+
+**Upstream Issue:** https://github.com/microsoft/TypeScript/issues/61037 (marked "Help Wanted", in Backlog, NOT FIXED as of Dec 2025)
+
+**Workaround:** Post-process `.d.ts` files with a script. See `script/fix-dts-extensions.cjs`.
+
+### 4. languageserver Tests Hang
+
+**Problem:** The languageserver tests hang indefinitely when running with the ESM configuration.
+
+**Status:** Not fully diagnosed. Tests pass on main branch but hang on ESM branch.
+
+**Possible causes:**
+- Jest ESM module resolution issues
+- Cross-package source mappings in jest.config.js
+- vscode-languageserver ESM compatibility issues
+- Specific test file causing hang (needs isolation testing)
+
+**Current Decision:** The languageserver package is **deferred** from this migration until the test hang issue is resolved. It will continue using the old configuration.
+
+**Investigation needed:**
+- Run individual test files to isolate the hanging test
+- Check if vscode-languageserver has ESM compatibility issues
+- Review jest configuration for problematic mappings
+- Try running with `--detectOpenHandles` flag
+
+---
+
+## Migration Status
+
+| Package | Tests | ESM Status |
+|---------|-------|------------|
+| expressions | 1068 | ✅ Migrated |
+| workflow-parser | 292 | ✅ Migrated |
+| languageservice | 452 | ✅ Migrated |
+| languageserver | 6 files | ⏸️ Deferred (test hang) |
+
+---
+
+## Required Configuration Changes
+
+### tsconfig.json (each migrated package)
+
+```json
+{
+  "compilerOptions": {
+    "module": "node16",
+    "moduleResolution": "node16",
+    "rewriteRelativeImportExtensions": true,
+    "lib": ["ES2022"],
+    "target": "ES2022"
+  }
+}
+```
+
+### jest.config.js (each migrated package)
+
+```javascript
+/** @type {import('ts-jest').JestConfigWithTsJest} */
+export default {
+  preset: "ts-jest/presets/default-esm",
+  moduleNameMapper: {
+    "^(\\.{1,2}/.*)\\.js$": "$1",
+    "^(\\.{1,2}/.*)\\.ts$": "$1",
+  },
+  transform: {
+    "^.+\\.tsx?$": [
+      "ts-jest",
+      {
+        useESM: true,
+        isolatedModules: true,
+      },
+    ],
+  },
+  moduleFileExtensions: ["ts", "js"],
+};
+```
+
+### Root package.json
+
+```json
+{
+  "overrides": {
+    "typescript": "5.8.3"
+  }
+}
+```
+
+### Each workspace package.json
+
+```json
+{
+  "devDependencies": {
+    "typescript": "^5.8.3",
+    "ts-jest": "^29.0.3"
+  }
+}
+```
+
+---
+
+## References
+
+- [TypeScript moduleResolution reference](https://www.typescriptlang.org/docs/handbook/modules/reference.html)
+- [TypeScript 5.7 rewriteRelativeImportExtensions](https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-7.html#path-rewriting-for-relative-paths)
+- [TypeScript .d.ts extension bug #61037](https://github.com/microsoft/TypeScript/issues/61037)
+- [Node.js ESM mandatory extensions](https://nodejs.org/api/esm.html#mandatory-file-extensions)
+- [ts-jest ESM support](https://kulshekhar.github.io/ts-jest/docs/guides/esm-support)
+- [Community fork that works](https://github.com/boxbuild-io/actions-languageservices/commit/077fb2b58dfd2cca3d6e3df1fdf9e26e75db24ae)

expressions/package.json

@@ -60,6 +60,6 @@
     "prettier": "^2.8.3",
     "rimraf": "^3.0.2",
     "ts-jest": "^29.0.3",
-    "typescript": "^4.7.4"
+    "typescript": "^5.8.3"
   }
 }

expressions/src/ast.ts

@@ -1,5 +1,5 @@
-import {ExpressionData} from "./data";
-import {Token} from "./lexer";
+import {ExpressionData} from "./data/index.js";
+import {Token} from "./lexer.js";
 
 export interface ExprVisitor<R> {
   visitLiteral(literal: Literal): R;

expressions/src/completion.test.ts

@@ -1,11 +1,11 @@
-import {complete, CompletionItem, trimTokenVector} from "./completion";
-import {DescriptionDictionary} from "./completion/descriptionDictionary";
-import {BooleanData} from "./data/boolean";
-import {Dictionary} from "./data/dictionary";
-import {StringData} from "./data/string";
-import {wellKnownFunctions} from "./funcs";
-import {FunctionDefinition, FunctionInfo} from "./funcs/info";
-import {Lexer, TokenType} from "./lexer";
+import {complete, CompletionItem, trimTokenVector} from "./completion.js";
+import {DescriptionDictionary} from "./completion/descriptionDictionary.js";
+import {BooleanData} from "./data/boolean.js";
+import {Dictionary} from "./data/dictionary.js";
+import {StringData} from "./data/string.js";
+import {wellKnownFunctions} from "./funcs.js";
+import {FunctionDefinition, FunctionInfo} from "./funcs/info.js";
+import {Lexer, TokenType} from "./lexer.js";
 
 const testContext = new Dictionary(
   {

expressions/src/completion.ts

@@ -1,11 +1,11 @@
-import {DescriptionPair} from "./completion/descriptionDictionary";
-import {Dictionary, isDictionary} from "./data/dictionary";
-import {ExpressionData} from "./data/expressiondata";
-import {Evaluator} from "./evaluator";
-import {wellKnownFunctions} from "./funcs";
-import {FunctionDefinition, FunctionInfo} from "./funcs/info";
-import {Lexer, Token, TokenType} from "./lexer";
-import {Parser} from "./parser";
+import {DescriptionPair} from "./completion/descriptionDictionary.js";
+import {Dictionary, isDictionary} from "./data/dictionary.js";
+import {ExpressionData} from "./data/expressiondata.js";
+import {Evaluator} from "./evaluator.js";
+import {wellKnownFunctions} from "./funcs.js";
+import {FunctionDefinition, FunctionInfo} from "./funcs/info.js";
+import {Lexer, Token, TokenType} from "./lexer.js";
+import {Parser} from "./parser.js";
 
 export type CompletionItem = {
   label: string;

expressions/src/completion/descriptionDictionary.test.ts

@@ -1,5 +1,5 @@
-import {StringData} from "../data";
-import {DescriptionDictionary} from "./descriptionDictionary";
+import {StringData} from "../data/index.js";
+import {DescriptionDictionary} from "./descriptionDictionary.js";
 
 describe("description dictionary", () => {
   it("pairs contains all values", () => {

expressions/src/completion/descriptionDictionary.ts

@@ -1,5 +1,5 @@
-import {Dictionary} from "../data/dictionary";
-import {ExpressionData, Kind, Pair} from "../data/expressiondata";
+import {Dictionary} from "../data/dictionary.js";
+import {ExpressionData, Kind, Pair} from "../data/expressiondata.js";
 
 export type DescriptionPair = Pair & {description?: string};
 

expressions/src/data/array.ts

@@ -1,4 +1,4 @@
-import {ExpressionData, ExpressionDataInterface, Kind, kindStr} from "./expressiondata";
+import {ExpressionData, ExpressionDataInterface, Kind, kindStr} from "./expressiondata.js";
 
 export class Array implements ExpressionDataInterface {
   private v: ExpressionData[] = [];

expressions/src/data/boolean.ts

@@ -1,4 +1,4 @@
-import {ExpressionDataInterface, Kind} from "./expressiondata";
+import {ExpressionDataInterface, Kind} from "./expressiondata.js";
 
 export class BooleanData implements ExpressionDataInterface {
   constructor(public readonly value: boolean) {}

expressions/src/data/dictionary.test.ts

@@ -1,5 +1,5 @@
-import {Dictionary} from "./dictionary";
-import {StringData} from "./string";
+import {Dictionary} from "./dictionary.js";
+import {StringData} from "./string.js";
 
 describe("dictionary", () => {
   it("pairs contains all values", () => {

expressions/src/data/dictionary.ts

@@ -1,4 +1,4 @@
-import {ExpressionData, ExpressionDataInterface, Kind, kindStr, Pair} from "./expressiondata";
+import {ExpressionData, ExpressionDataInterface, Kind, kindStr, Pair} from "./expressiondata.js";
 
 export class Dictionary implements ExpressionDataInterface {
   private keys: string[] = [];

expressions/src/data/expressiondata.ts

@@ -1,9 +1,9 @@
-import {Dictionary} from "./dictionary";
-import {Null} from "./null";
-import {Array} from "./array";
-import {StringData} from "./string";
-import {NumberData} from "./number";
-import {BooleanData} from "./boolean";
+import {Dictionary} from "./dictionary.js";
+import {Null} from "./null.js";
+import {Array} from "./array.js";
+import {StringData} from "./string.js";
+import {NumberData} from "./number.js";
+import {BooleanData} from "./boolean.js";
 
 export enum Kind {
   String = 0,

expressions/src/data/index.ts

@@ -1,9 +1,9 @@
-export {Array} from "./array";
-export {BooleanData} from "./boolean";
-export {Dictionary} from "./dictionary";
-export {ExpressionData, Kind} from "./expressiondata";
-export {Null} from "./null";
-export {NumberData} from "./number";
-export {replacer} from "./replacer";
-export {reviver} from "./reviver";
-export {StringData} from "./string";
+export {Array} from "./array.js";
+export {BooleanData} from "./boolean.js";
+export {Dictionary} from "./dictionary.js";
+export {ExpressionData, Kind} from "./expressiondata.js";
+export {Null} from "./null.js";
+export {NumberData} from "./number.js";
+export {replacer} from "./replacer.js";
+export {reviver} from "./reviver.js";
+export {StringData} from "./string.js";