DownloadRefDomainBackLinks

Command status: Active
Supported by OpenApps API: Yes
Supported by Internal/Reseller API: Yes
Possibly queued processing: Yes (Always)

Calls to this function will result in usage of the following subscription resources:

Resource Description

AnalysisResUnits

50000 + 20 per each ref domain (ref domains can be retrieved from GetIndexItemInfo 'RefDomains' column)

RetrievalResUnits

This resource will be decreased by actual number of rows of data retrieved (returned) by this function.

This function is designed to allow retrieval of large numbers of referring domains, beyond the 100,000 maximum of GetRefDomains or can return the top backlinks for each referring domain.

Parameter Description

cmd

Required: must be set to: DownloadRefDomainBackLinks

datasource

Optional - defaults to historic
Either: "fresh" - to query against fresh index, or "historic" - to query against historic index.

item

Required: index item that is being queried using same convention as for GetIndexItemInfo, examples:
http://www.example.com - URL
www.example.com - subdomain
example.com - root domain

MaxSourceURLsPerRefDomain

Optional: can be set to 1 to 10
Default: 10
Defines the maximum number of URLs per referring domain to be returned

Mode

Optional: can be set to 0 or 1
Default: 0
Includes or excludes deleted links, 0=show all links including deleted, 1=exclude deleted links from results

SkipIfAnalysisCostGreaterThan

Analysis will be automatically aborted for any index item that will have greater analysis cost than specified. This is useful to prevent human errors in trying to download backlinks for items that are too large.

Default: 1000000

NotifyURL

Important: notification URL must be accessible from outside your intranet - do not specify internal servers that can't be assessed from our servers, if you do then you will never get the notification you expect!

Optional: if specified this URL (with HTTP/HTTPS protocols only) will be requested to notify you that the download has been fully prepared. The URL you provide can contain query string parameters to help you identify the download request that was made, we will also substitute (if they are present) the following macro variables (case sensitive): %%DOWNLOAD_FILE%% - will be changed to the download filename %%DOWNLOAD_FILE_LOCATION%% - will be changed to the download filename location using PublicDownloadLocation variable below

Your notify URL should respond with single piece of data: OK (2 characters - no HTML) to indicate that you have successfully received this notification. Any other response (including server failures on your side) will be treated as an error. In the case of an error, the notification URL will be called again a number of times using exponential backoff before failing.

Example of notification url: http://www.example.com/mjseonotify.php (this URL doesn't actually exist!)

UploadToFTP

Important: this FTP URL must be accessible from outside your intranet - do not specify internal servers that can't be assessed from our side: if you do then you will never get the upload!

Optional: if not specified prepared file with backlinks will be uploaded to majestic.com, this behaviour can be overridden by specifying your own FTP server using appropriate URI format such as:

ftp://username:password@yourftpserver.com/public_html/uploads/ with the relative path of the upload directory or ftp://username:password@yourftpserver.com//home/user/public_html/uploads/ with the absolute path of the upload directory.

Make sure that the user specified is allowed to write files in the directory that you designated for these uploads. If a trailing filename is specified, this will be prepended to the output filename when it is upload to your server.

If not specified the upload will be made to www.majestic.com from where you will be able to download the file.

PublicDownloadLocation

Optional: by default prepared backlinks file will be available from URL: http://downloads.majestic.com (note: directory listing is denied by design there) - IF you use alternative FTP location and know which public URL would correspond to it then you are advised to supply it here unless you are planning to analyse the data locally.

Sample query and response

This is a protocol-level example query that uses a special URL that was overridden to have zero cost of analysis (you will need to use your own API_KEY to analyse other urls):

https://api.majestic.com/api/xml?app_api_key=API_KEY&cmd=DownloadRefDomainBackLinks&item=majestic.com

Sample XML response:



https://api.majestic.com/api/json?app_api_key=API_KEY&cmd=DownloadRefDomainBackLinks&item=majestic.com

Sample JSON response:



This response indicates that the request was queued for asynchronous processing. Note: JobID value is returned in this case. This response indicates that this request has already been requested recently and data files were prepared for it - see GetDownloadsList command how to check for those data files. Sample check query for this particular request data files (note: DownloadJobID value will be different - take it from XML returned by previous call):
https://api.majestic.com/api/xml?app_api_key=API_KEY&cmd=GetDownloadsList&DownloadJobID=23C65F56E42833BE94305BCA6B10320F



https://api.majestic.com/api/json?app_api_key=API_KEY&cmd=GetDownloadsList&DownloadJobID=BE82255B70F2CD1DCC06762FA68BE9B9

Sample JSON response:

The response indicates where data files are located. Please note that it is possible to supply your own FTP location for such uploads and also it is highly recommended to use NotifyURL functionality that will call your web application to confirm that processing of such long running request is over.


Data file can now be downloaded from PublicDownloadLocation shown in XML above. The format of that file is CSV (Comma Separated Values) with UTF-8 encoding format. The first line of the file is always the header. Target URL - URL where backlink is pointing to
Target ACRank - ACRank of Target URL
Source URL - Source URL (backlink)
Source ACRank - ACRank of Source URL
Anchor Text - anchor text used in linking, for images it will be text used in ALT="" part of the tag and for Mentions it will be text used (ie: example.com)
Source Crawl Date - the date when backlink was last (most recently) crawled
Source First Found Date - the date when backlink was first found on source page (historical data is more meaningful in this context because fresh index only covers last 30 days of crawl)
FlagNoFollow - if set to + then source URL was marked as nofollow
FlagImageLink - if set to + then source URL was image
FlagRedirect - if set to + then source URL was redirect
FlagFrame - if set to + then source URL was used in FRAME or IFRAME
FlagOldCrawl - if set to + then source URL was found to be deleted (removed)
FlagAltText - if set to + then source URL was taken from TITLE part of an A tag
FlagMention - if set to + then source URL was actually text mention
SourceCitationFlow - Citation Flow of the Source URL
SourceTrustFlow - Trust Flow of the Source URL
TargetCitationFlow - Citation Flow of the Target URL
TargetTrustFlow - Trust Flow of the Target URL
SourceTopicalTrustFlow_Topic_0 - Highest ranking trust flow topic for Source URL (if available) SourceTopicalTrustFlow_Value_0 - Highest ranking trust flow topic score for Source URL (if available) TargetTopicalTrustFlow_Topic_0 - Highest ranking trust flow topic for Target URL (if available) TargetTopicalTrustFlow_Value_0 - Highest ranking trust flow topic score for Target URL (if available)


Sample API request to delete this job:
https://api.majestic.com/api/xml?app_api_key=API_KEY&cmd=DeleteDownloads&DownloadJobIDs=23C65F56E42833BE94305BCA6B10320F

Sample XML response indicating that the job was deleted successfully:



Sample API request to delete this job:
https://api.majestic.com/api/json?app_api_key=API_KEY&cmd=DeleteDownloads&DownloadJobIDs=BE82255B70F2CD1DCC06762FA68BE9B9

Sample JSON response indicating that the job was deleted successfully:

Common problems

Below you can see a number of common problems that were experienced by our customers.

Problem: Calling to analyse very large domains (like google.com) can quickly use up available resources.
Solution: Do not analyse large domain unless you really mean it - each call to this function results in server processing data and this reduces your resource allowance.
Problem: calling analysis of the same domain many times over with slightly different parameters can quickly use up available resources.
Solution: call this function once to get all data you need and then on your end you can make all sort of separate calls to the data.

For more information about access to the Majestic API suite, visit our Plans & Pricing page.