Function loadModel

  • Load a model spec or try to guess it from a prebuilt-id.

    Parameters

    • specOrId: string | ModelSpec
    • Optional report: ((loadReport) => void)
        • (loadReport): void

        • Parameters

          Returns void

    • targetDevice: TargetDevice = TargetDevice.GPU

    Returns Promise<LoadedModel>

    Remarks

    Passing a prebuilt-id currently only works for Llama 2 models due to wasm naming differences. Check here for some prebuilt-ids: https://github.com/mlc-ai/binary-mlc-llm-libs

    Example

    const model = await loadModel('Llama-2-7b-chat-hf-q4f32_1')

Settings

Theme

Generated using TypeDoc