Class HNLoader

A class that extends the CheerioWebBaseLoader class. It represents a loader for loading web pages from the Hacker News website.

Hierarchy (view full)

Constructors

constructor

Properties

caller timeout webPath selector? textDecoder?

Methods

load loadAndSplit scrape imports scrapeAll

Constructors

constructor

Properties

caller

caller: AsyncCaller

timeout

timeout: number

webPath

webPath: string

Optional selector

selector?: `.${number}${string}` | `.i${string}` | `.a${string}` | `.b${string}` | `.c${string}` | `.d${string}` | `.e${string}` | `.f${string}` | `.g${string}` | `.h${string}` | `.j${string}` | `.k${string}` | `.l${string}` | `.m${string}` | `.n${string}` | `.o${string}` | `.p${string}` | `.q${string}` | `.r${string}` | `.s${string}` | `.t${string}` | `.u${string}` | `.v${string}` | `.w${string}` | `.x${string}` | `.y${string}` | `.z${string}` | `.I${string}` | `.A${string}` | `.B${string}` | `.C${string}` | `.D${string}` | `.E${string}` | `.F${string}` | `.G${string}` | `.H${string}` | `.J${string}` | `.K${string}` | `.L${string}` | `.M${string}` | `.N${string}` | `.O${string}` | `.P${string}` | `.Q${string}` | `.R${string}` | `.S${string}` | `.T${string}` | `.U${string}` | `.V${string}` | `.W${string}` | `.X${string}` | `.Y${string}` | `.Z${string}` | `#${number}${string}` | `#i${string}` | `#a${string}` | `#b${string}` | `#c${string}` | `#d${string}` | `#e${string}` | `#f${string}` | `#g${string}` | `#h${string}` | `#j${string}` | `#k${string}` | `#l${string}` | `#m${string}` | `#n${string}` | `#o${string}` | `#p${string}` | `#q${string}` | `#r${string}` | `#s${string}` | `#t${string}` | `#u${string}` | `#v${string}` | `#w${string}` | `#x${string}` | `#y${string}` | `#z${string}` | `#I${string}` | `#A${string}` | `#B${string}` | `#C${string}` | `#D${string}` | `#E${string}` | `#F${string}` | `#G${string}` | `#H${string}` | `#J${string}` | `#K${string}` | `#L${string}` | `#M${string}` | `#N${string}` | `#O${string}` | `#P${string}` | `#Q${string}` | `#R${string}` | `#S${string}` | `#T${string}` | `#U${string}` | `#V${string}` | `#W${string}` | `#X${string}` | `#Y${string}` | `#Z${string}` | `:${number}${string}` | `:i${string}` | `:a${string}` | `:b${string}` | `:c${string}` | `:d${string}` | `:e${string}` | `:f${string}` | `:g${string}` | `:h${string}` | `:j${string}` | `:k${string}` | `:l${string}` | `:m${string}` | `:n${string}` | `:o${string}` | `:p${string}` | `:q${string}` | `:r${string}` | `:s${string}` | `:t${string}` | `:u${string}` | `:v${string}` | `:w${string}` | `:x${string}` | `:y${string}` | `:z${string}` | `:I${string}` | `:A${string}` | `:B${string}` | `:C${string}` | `:D${string}` | `:E${string}` | `:F${string}` | `:G${string}` | `:H${string}` | `:J${string}` | `:K${string}` | `:L${string}` | `:M${string}` | `:N${string}` | `:O${string}` | `:P${string}` | `:Q${string}` | `:R${string}` | `:S${string}` | `:T${string}` | `:U${string}` | `:V${string}` | `:W${string}` | `:X${string}` | `:Y${string}` | `:Z${string}` | `|${number}${string}` | `|i${string}` | `|a${string}` | `|b${string}` | `|c${string}` | `|d${string}` | `|e${string}` | `|f${string}` | `|g${string}` | `|h${string}` | `|j${string}` | `|k${string}` | `|l${string}` | `|m${string}` | `|n${string}` | `|o${string}` | `|p${string}` | `|q${string}` | `|r${string}` | `|s${string}` | `|t${string}` | `|u${string}` | `|v${string}` | `|w${string}` | `|x${string}` | `|y${string}` | `|z${string}` | `|I${string}` | `|A${string}` | `|B${string}` | `|C${string}` | `|D${string}` | `|E${string}` | `|F${string}` | `|G${string}` | `|H${string}` | `|J${string}` | `|K${string}` | `|L${string}` | `|M${string}` | `|N${string}` | `|O${string}` | `|P${string}` | `|Q${string}` | `|R${string}` | `|S${string}` | `|T${string}` | `|U${string}` | `|V${string}` | `|W${string}` | `|X${string}` | `|Y${string}` | `|Z${string}` | `>${number}${string}` | `>i${string}` | `>a${string}` | `>b${string}` | `>c${string}` | `>d${string}` | `>e${string}` | `>f${string}` | `>g${string}` | `>h${string}` | `>j${string}` | `>k${string}` | `>l${string}` | `>m${string}` | `>n${string}` | `>o${string}` | `>p${string}` | `>q${string}` | `>r${string}` | `>s${string}` | `>t${string}` | `>u${string}` | `>v${string}` | `>w${string}` | `>x${string}` | `>y${string}` | `>z${string}` | `>I${string}` | `>A${string}` | `>B${string}` | `>C${string}` | `>D${string}` | `>E${string}` | `>F${string}` | `>G${string}` | `>H${string}` | `>J${string}` | `>K${string}` | `>L${string}` | `>M${string}` | `>N${string}` | `>O${string}` | `>P${string}` | `>Q${string}` | `>R${string}` | `>S${string}` | `>T${string}` | `>U${string}` | `>V${string}` | `>W${string}` | `>X${string}` | `>Y${string}` | `>Z${string}` | `+${number}${string}` | `+i${string}` | `+a${string}` | `+b${string}` | `+c${string}` | `+d${string}` | `+e${string}` | `+f${string}` | `+g${string}` | `+h${string}` | `+j${string}` | `+k${string}` | `+l${string}` | `+m${string}` | `+n${string}` | `+o${string}` | `+p${string}` | `+q${string}` | `+r${string}` | `+s${string}` | `+t${string}` | `+u${string}` | `+v${string}` | `+w${string}` | `+x${string}` | `+y${string}` | `+z${string}` | `+I${string}` | `+A${string}` | `+B${string}` | `+C${string}` | `+D${string}` | `+E${string}` | `+F${string}` | `+G${string}` | `+H${string}` | `+J${string}` | `+K${string}` | `+L${string}` | `+M${string}` | `+N${string}` | `+O${string}` | `+P${string}` | `+Q${string}` | `+R${string}` | `+S${string}` | `+T${string}` | `+U${string}` | `+V${string}` | `+W${string}` | `+X${string}` | `+Y${string}` | `+Z${string}` | `~${number}${string}` | `~i${string}` | `~a${string}` | `~b${string}` | `~c${string}` | `~d${string}` | `~e${string}` | `~f${string}` | `~g${string}` | `~h${string}` | `~j${string}` | `~k${string}` | `~l${string}` | `~m${string}` | `~n${string}` | `~o${string}` | `~p${string}` | `~q${string}` | `~r${string}` | `~s${string}` | `~t${string}` | `~u${string}` | `~v${string}` | `~w${string}` | `~x${string}` | `~y${string}` | `~z${string}` | `~I${string}` | `~A${string}` | `~B${string}` | `~C${string}` | `~D${string}` | `~E${string}` | `~F${string}` | `~G${string}` | `~H${string}` | `~J${string}` | `~K${string}` | `~L${string}` | `~M${string}` | `~N${string}` | `~O${string}` | `~P${string}` | `~Q${string}` | `~R${string}` | `~S${string}` | `~T${string}` | `~U${string}` | `~V${string}` | `~W${string}` | `~X${string}` | `~Y${string}` | `~Z${string}` | `[${number}${string}` | `[i${string}` | `[a${string}` | `[b${string}` | `[c${string}` | `[d${string}` | `[e${string}` | `[f${string}` | `[g${string}` | `[h${string}` | `[j${string}` | `[k${string}` | `[l${string}` | `[m${string}` | `[n${string}` | `[o${string}` | `[p${string}` | `[q${string}` | `[r${string}` | `[s${string}` | `[t${string}` | `[u${string}` | `[v${string}` | `[w${string}` | `[x${string}` | `[y${string}` | `[z${string}` | `[I${string}` | `[A${string}` | `[B${string}` | `[C${string}` | `[D${string}` | `[E${string}` | `[F${string}` | `[G${string}` | `[H${string}` | `[J${string}` | `[K${string}` | `[L${string}` | `[M${string}` | `[N${string}` | `[O${string}` | `[P${string}` | `[Q${string}` | `[R${string}` | `[S${string}` | `[T${string}` | `[U${string}` | `[V${string}` | `[W${string}` | `[X${string}` | `[Y${string}` | `[Z${string}` | `${number}${string}` | `i${string}` | `a${string}` | `b${string}` | `c${string}` | `d${string}` | `e${string}` | `f${string}` | `g${string}` | `h${string}` | `j${string}` | `k${string}` | `l${string}` | `m${string}` | `n${string}` | `o${string}` | `p${string}` | `q${string}` | `r${string}` | `s${string}` | `t${string}` | `u${string}` | `v${string}` | `w${string}` | `x${string}` | `y${string}` | `z${string}` | `I${string}` | `A${string}` | `B${string}` | `C${string}` | `D${string}` | `E${string}` | `F${string}` | `G${string}` | `H${string}` | `J${string}` | `K${string}` | `L${string}` | `M${string}` | `N${string}` | `O${string}` | `P${string}` | `Q${string}` | `R${string}` | `S${string}` | `T${string}` | `U${string}` | `V${string}` | `W${string}` | `X${string}` | `Y${string}` | `Z${string}`

Optional textDecoder

textDecoder?: TextDecoder

Methods

load

  • load(): Promise<Document[]>

  • An asynchronous method that loads the web page. If the webPath includes "item", it calls the loadComments() method to load the comments from the web page. Otherwise, it calls the loadResults() method to load the results from the web page.

    Returns Promise<Document[]>

    A Promise that resolves to an array of Document instances.

loadAndSplit

  • loadAndSplit(splitter?): Promise<Document[]>

  • Loads the documents and splits them using a specified text splitter.

    Parameters

    • splitter: TextSplitter = ...

    Returns Promise<Document[]>

    A Promise that resolves with an array of Document instances, each split according to the provided TextSplitter.

scrape

Static imports

  • imports(): Promise<{
        load: ((content, options?, isDocument?) => CheerioAPI);
    }>

  • A static method that dynamically imports the Cheerio library and returns the load function. If the import fails, it throws an error.

    Returns Promise<{
        load: ((content, options?, isDocument?) => CheerioAPI);
    }>

    A Promise that resolves to an object containing the load function from the Cheerio library.

Static scrapeAll

  • scrapeAll(urls, caller, timeout, textDecoder?, options?): Promise<CheerioAPI[]>

  • Fetches web documents from the given array of URLs and loads them using Cheerio. It returns an array of CheerioAPI instances.

    Parameters

    • urls: string[]

      An array of URLs to fetch and load.

    • caller: AsyncCaller
    • timeout: undefined | number
    • Optional textDecoder: TextDecoder
    • Optional options: CheerioOptions

    Returns Promise<CheerioAPI[]>

    A Promise that resolves to an array of CheerioAPI instances.

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc