Apify Scraper for AI Crawling

with the apify scraper for ai crawling app in make, you can extract valuable text content from websites to feed ai models, llm applications, vector databases, or rag (retrieval augmented generation) pipelines requirements to use the apify scraper for ai crawling app in {{product name}} , you must have an apify account you can sign up using your email, gmail, or github account connect apify scraper for ai crawling and {{product name}} to get started, you must first create a connection between apify scraper for ai crawling and {{product name}} , allowing the two services to communicate make provides two ways to connect the apify scraper for ai crawling app apify scraper for ai crawling docid\ an7xnaauaq1czu7zwgumh apify scraper for ai crawling docid\ an7xnaauaq1czu7zwgumh establish a connection with your apify account to establish the connection log in to your {{product name}} account, add an apify scraper for ai crawling module to your {{scenario singular lowercase}} , and click create a connection optional in the ​​ connection name​​ field, enter a name for the connection click save if prompted, authenticate your account and confirm access you've successfully created the connection and can now use the apify scraper for ai crawling app in your {{scenario plural lowercase}} if you want to make changes to your connection in the future, follow the steps here https //help make com/connect an application#tn2pj establish a connection with api token before you create the connection in {{product name}} , you should first obtain an api token in your apify account to create the connection log in to your apify console in the left sidebar, click settings click api & integrations copy the personal api token value and store it in a safe place alternatively, you can also create a new api token with multiple customizable permissions by clicking on + create a new token log in to your {{product name}} account, add an apify scraper for ai crawling module to your {{scenario singular lowercase}} , and click create a connection optional in the ​​ connection name​​ field, enter a name for the connection in the api token field, enter the api token you copied from your apify account click save if prompted, authenticate your account and confirm access you've successfully created the connection and can now use the apify scraper for ai crawling app in your {{scenario plural lowercase}} if you want to make changes to your connection in the future, follow the steps here https //help make com/connect an application#tn2pj apify scraper for ai crawling modules after connecting to the apify scraper for ai crawling app, you can use the following types of modules to build your {{scenario plural lowercase}} searches crawl a website (standard settings) for each crawled web page, you'll receive page metadata url, title, description, canonical url cleaned text content the main article content with irrelevant elements removed markdown formatting structured content with headers, lists, links, and other formatting preserved crawl information loaded url, referrer url, timestamp, http status optional file downloads pdfs, docs, and other linked documents crawl a website (advanced settings) in addition to the standard output fields, advanced settings provides multiple format options content in markdown, html, or plain text debug information detailed extraction diagnostics and snapshots html transformations results from different content extraction algorithms file storage options flexible storage for html, screenshots, or downloaded files templates you can look for apify scraper for ai crawling templates in make's template gallery , where you'll find thousands of pre created {{scenario plural lowercase}} apify scraper for ai crawling resources apify api documentation apify ai crawling actor apify documentation