eu.dnetlib.bioschemas.api.scraper

Class ServiceScraper

java.lang.Object

hwu.elixir.scrape.scraper.ScraperCore

hwu.elixir.scrape.scraper.ScraperFilteredCore

eu.dnetlib.bioschemas.api.scraper.ServiceScraper

public class ServiceScraper
extends hwu.elixir.scrape.scraper.ScraperFilteredCore

Provides the actual scraping functionality. Scrapes a given URL, converts into NQuads and writes to a file (name derived from URL). If the file already exists it will be overwritten.

See Also:

ScraperFilteredCore

Field Summary

Fields inherited from class hwu.elixir.scrape.scraper.ScraperCore

properties

Constructor Summary

Constructors

Constructor and Description
ServiceScraper()

Method Summary

Modifier and Type	Method and Description
boolean	scrape(String url, Long contextCounter, String outputFolderName, String fileName, StatusOfScrape status) Orchestrates the process of scraping a site before converting the extracted triples to NQuads and writing to a file.
protected String	wrapHTMLExtraction(String url)

Methods inherited from class hwu.elixir.scrape.scraper.ScraperFilteredCore

fixAJsonLdArray, fixAllJsonLdBlocks, fixASingleJsonLdBlock, fixASingleJsonLdObject, fixASingleJSONLdObject, injectId, iriGenerator, processTriples, scrape, swapJsonLdMarkup

Methods inherited from class hwu.elixir.scrape.scraper.ScraperCore

createModelFromNTriples, displayResult, fixAny23WeirdIssues, fixObject, fixPredicate, fixURL, getHtml, getHtmlViaSelenium, getOnlyUnfilteredJSONLDFromHtml, getTripelsInNTriplesRDF4J, getTriplesInJSONLD, getTriplesInNTriples, shutdown, wrapHTMLExtractionStatic

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ServiceScraper

public ServiceScraper()

Method Detail

scrape

public boolean scrape(String url,
                      Long contextCounter,
                      String outputFolderName,
                      String fileName,
                      StatusOfScrape status)
               throws hwu.elixir.scrape.exceptions.FourZeroFourException,
                      hwu.elixir.scrape.exceptions.JsonLDInspectionException,
                      hwu.elixir.scrape.exceptions.CannotWriteException,
                      hwu.elixir.scrape.exceptions.MissingMarkupException

Orchestrates the process of scraping a site before converting the extracted triples to NQuads and writing to a file.

Parameters:

url - Site to be scraped

contextCounter - Number used to generate the named graph/context and the URLs used to replace blank nodes.

outputFolderName - Location to which the NQuads will be written

Returns:

True if success; false otherwise

Throws:

hwu.elixir.scrape.exceptions.FourZeroFourException

hwu.elixir.scrape.exceptions.JsonLDInspectionException

hwu.elixir.scrape.exceptions.CannotWriteException

hwu.elixir.scrape.exceptions.MissingMarkupException

wrapHTMLExtraction

protected String wrapHTMLExtraction(String url)
                             throws hwu.elixir.scrape.exceptions.FourZeroFourException

Overrides:

wrapHTMLExtraction in class hwu.elixir.scrape.scraper.ScraperCore

Throws:

hwu.elixir.scrape.exceptions.FourZeroFourException