curl - Using R to accept cookies to download a PDF file -

-

i'm getting stuck on cookies when trying download pdf.

for example, if have doi pdf document on archaeology data service, resolve this landing page embedded link in pdf redirects this other link.

library(httr) handle resolving doi , can extract pdf url landing page using library(xml) i'm stuck @ getting pdf itself.

if this:

download.file("http://archaeologydataservice.ac.uk/archiveds/archivedownload?t=arch-1352-1/dissemination/pdf/dyfed/gl44004.pdf", destfile = "tmp.pdf")

then receive html file same http://archaeologydataservice.ac.uk/myads/

trying answer @ how use r download zipped file ssl page requires cookies leads me this:

library(httr)  terms <- "http://archaeologydataservice.ac.uk/myads/copyrights" download <- "http://archaeologydataservice.ac.uk/archiveds/archivedownload" values <- list(agree = "yes", t = "arch-1352-1/dissemination/pdf/dyfed/gl44004.pdf")  # accept terms on form, # generating appropriate cookies  post(terms, body = values) get(download, query = values)  # download file (this take while)  resp <- get(download, query = values)  # write content of download binary file  writebin(content(resp, "raw"), "c:/temp/thefile.zip")

but after post , get functions html of same cookie page got download.file:

> get(download, query = values) response [http://archaeologydataservice.ac.uk/myads/copyrights?from=2f6172636869766544532f61726368697665446f776e6c6f61643f61677265653d79657326743d617263682d313335322d3125324664697373656d696e6174696f6e2532467064662532464479666564253246474c34343030342e706466]   date: 2016-01-06 00:35   status: 200   content-type: text/html;charset=utf-8   size: 21 kb <?xml version='1.0' encoding='utf-8' ?> <!doctype html public "-//w3c//dtd xhtml 1.0 transitional//en" "h... <html xmlns="http://www.w3.org/1999/xhtml" lang="en">         <head>             <meta http-equiv="content-type" content="text/html; c...               <title>archaeology data service:  myads</title>              <link href="http://archaeologydataservice.ac.uk/css/u... ...

looking @ http://archaeologydataservice.ac.uk/about/cookies seems cookie situation @ site complicated. seems kind of cookie complexity not unusual uk data providers: automating login uk data service website in r rcurl or httr

how can use r past cookies on website?

your plea on ropensci has been heard!

there's lots of javascript between pages makes annoying try decipher via httr + rvest. try rselenium. worked on os x 10.11.2, r 3.2.3 & firefox loaded.

library(rselenium)  # check if sever present, if not, server checkforserver()  # server going startserver()  dir.create("~/justcreateddir") setwd("~/justcreateddir")  # need pdfs download instead of display in-browser prefs <- makefirefoxprofile(list(   `browser.download.folderlist` = as.integer(2),   `browser.download.dir` = getwd(),   `pdfjs.disabled` = true,   `plugin.scan.plid.all` = false,   `plugin.scan.acrobat` = "99.0",   `browser.helperapps.neverask.savetodisk` = 'application/pdf' )) # browser going dr <- remotedriver$new(extracapabilities=prefs) dr$open()  # go page pdf dr$navigate("http://archaeologydataservice.ac.uk/archives/view/greylit/details.cfm?id=17755")  # find pdf link , "hit enter" pdf_elem <- dr$findelement(using="css selector", "a.dlb3") pdf_elem$sendkeystoelement(list("\ue007"))  # find accept button , "hit enter" # save pdf default downloads directory accept_elem <- dr$findelement(using="css selector", "a[id$='agreebutton']") accept_elem$sendkeystoelement(list("\ue007"))

now wait download complete. r console not busy while downloads, easy close session accidently, before download has completed.

# close session dr$close()

Comments

Post a Comment

Popular posts from this blog

how to insert data php javascript mysql with multiple array session 2 -

-
i have method method discussed , successful data entered 1 when more 1 can not database this programs my php form : <form class="form-horizontal" id="form-produk"> <div class="control-group" > <label class="control-label" for="kelas">kategori</label> <div class="controls"> <select class='form-control' name='id_kategori'> <?php $tampil=mysql_query("select * kategori order nama_kategori"); if ($id_kat==0){ echo "<option value=0 selected>- pilih kategori -</option>"; } while($w=mysql_fetch_array($tampil)){ if ($id_kat==$w[id_kategori]){ echo "<option value=$w[id_kategori] selected>$w[nama_kategori]</option>"; } else{ echo "<option value=$w[id_kategori]>$...
Read more

multithreading - Exception in Application constructor -

-
i wanna write snake game javafx , there exception don't know , want know how fix it. ( know it's not complete yet ) and want know , class extends application ( start override ) main in other programs? see , here not static void main bc didn't need, if want add main shoud do? it's exeption... exception in application constructor exception in thread "main" java.lang.nosuchmethodexception: main_snake.main([ljava.lang.string;) @ java.lang.class.getmethod(class.java:1819) @ com.intellij.rt.execution.application.appmain.main(appmain.java:125) and code : import javafx.animation.animationtimer; import javafx.application.application; import javafx.event.eventhandler; import javafx.scene.scene; import javafx.scene.canvas.canvas; import javafx.scene.canvas.graphicscontext; import javafx.scene.input.keyevent; import javafx.scene.layout.borderpane; import javafx.scene.paint.color; import javafx.stage.stage; import java.util.arraylist; /** * creat...
Read more

windows - CertCreateCertificateContext returns CRYPT_E_ASN1_BADTAG / 8009310b -

-
i realize similar post others (e.g. this one ), there details missing posts might significant case. to start with, here's simplified program: #include "stdafx.h" #include <windows.h> #include <wincrypt.h> int _tmain(int argc, _tchar* argv[]) { // usage: certextract certpath char keyfile[] = "c:\\certificates\\public.crt"; byte lp[65536]; security_attributes sa; handle hkeyfile; dword bytes; pccert_context certcontext; sa.nlength = sizeof(sa); sa.lpsecuritydescriptor = null; sa.binherithandle = false; hkeyfile = createfile(keyfile, generic_read, file_share_read, &sa, open_existing, file_attribute_normal, null); if (hkeyfile) { if (readfile(hkeyfile, lp, getfilesize(hkeyfile, null), &bytes, null) && bytes > 0) { certcontext = certcreatecertificatecontext(x509_asn_encoding, lp, bytes); if (certcontext) { printf("yay...
Read more