I am trying to create a way to automate using IDrive's sync functionality. IDrive at one time had two types of accounts, backup and sync. As such there was a backup API and a sync API. But now the larger backup accounts have sync functionality included but I still am unable to use the official sync API because I do not have a dedicated sync account. So I looked at Mr. Inspector to see what was going on and it this is what I think they are doing.
You go to https://www.idrive.com/idrive/loginForm and there is a standard username and password form but it has three hidden fields: csrftoken, support, and from. When you populate your username and password a POST to https://www.idrive.com/idrive/validateLogin happens with the form data being the following in this order:
support: currently an empty string csrftoken: random string from the loginForm page email: username or email address password: password from: always seems to be 'singleLogin'
Now loginForm also generates a Tomcat JSESSIONID cookie which I am also capturing. When the browser makes the request to /validateLogin the JSESSIONID is sent as a Cookie in the request header. The /validateLogin page returns some new cookies which presumably are used as your auth.
I have replicated the entire transaction in Python and Ruby scripts, including all headers and cookies, but when I post to /validateLogin I get 302'd to /logout and I'm not sure why. I am capturing the values of the hidden fields and including them in my POST. I cannot see anything different between the actual browser and my scripts.
I have captured the entire transaction using LiveHTTPHeaders.
Here is the request to and response from /loginForm
https://www.idrive.com/idrive/login/loginForm
Host: www.idrive.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
GET: HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Jul 2018 16:47:42 GMT
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: JSESSIONID=E0C3A608B7042A9F792ADB1806394C00.tomcat8; Path=/idrive/; Secure; HttpOnly
REMEMBER_ME_COOKIE=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/
REMEMBER_ME_COOKIE=""; Domain=.www.idrive.com; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/
CURRENTPATH=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/
FILENAMEWIDTH=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/
FILESIZEWIDTH=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/
FILELMDWIDTH=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/
THUMBVIEW=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/
CHECKINSESSION=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/
Content-Language: en-US
X-FRAME-OPTIONS: SAMEORIGIN, SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=15768000
Content-Encoding: gzip
Here is the request to and response from /validateLogin
https://www.idrive.com/idrive/login/validateLogin
Host: www.idrive.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://www.idrive.com/idrive/login/loginForm
Content-Type: application/x-www-form-urlencoded
Content-Length: 115
Cookie: JSESSIONID=E0C3A608B7042A9F792ADB1806394C00.tomcat8
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
support=&csrftoken=EOc2ByDQ6KKM8B0Im47YwgLu3zaDRrGBPw8w2mXp&email=MYUSERNAME&password=MYPASSWORD&from=singleLogin
POST: HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Jul 2018 16:47:54 GMT
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Set-Cookie: remoteManageCookie=RANDOMSTRING; Expires=Mon, 14-Jan-2019 16:47:54 GMT; Path=/
CHECKINSESSION=MYUSERNAME; Path=/
EVS_SERVER=evsweb2590.idrive.com; Path=/
EVS_SERVER_SYNC=evsweb20.idrivesync.com; Path=/
X-FRAME-OPTIONS: SAMEORIGIN, SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=15768000
Content-Encoding: gzip
And here is the Ruby script I am using to try and automate the process
#!/usr/bin/env ruby
require 'pp'
require 'uri'
require 'persistent_http'
require 'nokogiri'
require 'open-uri'
def process_cookies(string)
string.split('; ')
end
def find_hidden(body)
page = Nokogiri::HTML(body)
login_form = page.css('form#loginForm')
inputs = login_form.css('input')
csrftoken = inputs.select{|i| i['name'] == 'csrftoken'}.first['value']
support = inputs.select{|i| i['name'] == 'support'}.first['value']
from = inputs.select{|i| i['name'] == 'from'}.first['value']
return csrftoken || nil, support || nil, from || nil
end
persistent_http = PersistentHTTP.new(
:name => 'idrive',
:pool_size => 10,
:pool_timeout => 5,
:warn_timeout => 0.25,
:force_retry => true,
:use_ssl => true,
:host => 'www.idrive.com',
)
req = Net::HTTP::Get.new('/idrive/login/loginForm')
res = persistent_http.request(req)
puts '*** /loginForm request headers ***'
pp req.to_hash
puts ''
puts '*** /loginForm response headers ***'
pp res.to_hash
puts ''
cookies = process_cookies(res.response['set-cookie'])
jsession_id = cookies.select{|c| c =~ /^JSESSIONID/}.first || nil
csrftoken, support, from = find_hidden(res.body)
username, password = 'MYUSERNAME', 'MYPASSWORD'
form_data = {
'support' => support,
'csfrtoken' => csrftoken,
'email' => username,
'password' => password,
'from' => from,
}
req = Net::HTTP::Post.new('/idrive/login/validateLogin')
req.set_form_data(form_data)
req['Accept'] = 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
req['Accept-Encoding'] = 'gzip, deflate, br'
req['Accept-Language'] = 'en-US,en;q=0.9'
req['Cache-Control'] = 'max-age=0'
req['Connection'] = 'keep-alive'
req['Content-Length'] = req.body.length
req['Content-Type'] = 'application/x-www-form-urlencoded'
req['Cookie'] = jsession_id
req['Host'] = 'www.idrive.com'
req['Origin'] = 'https://www.idrive.com'
req['Referer'] = 'https://www.idrive.com/idrive/login/loginForm'
req['Upgrade-Insecure-Requests'] = 1
req['User-Agent'] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0'
res = persistent_http.request(req)
puts '*** /validateLogin request headers ***'
pp req.to_hash
puts ''
puts '*** /validateLogin POST data'
puts req.body
puts ''
puts '*** /validateLogin response headers ***'
pp res.to_hash
Finally, here is the output from the above script
*** /loginForm request headers ***
{"accept-encoding"=>["gzip;q=1.0,deflate;q=0.6,identity;q=0.3"],
"accept"=>["*/*"],
"user-agent"=>["Ruby"],
"connection"=>["keep-alive"],
"keep-alive"=>["30"],
"host"=>["www.idrive.com"]}
*** /loginForm response headers ***
{"server"=>["nginx"],
"date"=>["Wed, 18 Jul 2018 17:05:41 GMT"],
"content-type"=>["text/html;charset=UTF-8"],
"transfer-encoding"=>["chunked"],
"connection"=>["keep-alive"],
"x-frame-options"=>["SAMEORIGIN", "SAMEORIGIN"],
"set-cookie"=>
["JSESSIONID=5BEF2E080C765C10CA285812B29C1FD2.tomcat8; Path=/idrive/; Secure; HttpOnly",
"REMEMBER_ME_COOKIE=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/",
"REMEMBER_ME_COOKIE=\"\"; Domain=.www.idrive.com; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/",
"CURRENTPATH=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/",
"FILENAMEWIDTH=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/",
"FILESIZEWIDTH=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/",
"FILELMDWIDTH=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/",
"THUMBVIEW=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/idrive/",
"CHECKINSESSION=\"\"; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/"],
"content-language"=>["en-US"],
"x-xss-protection"=>["1; mode=block"],
"x-content-type-options"=>["nosniff"],
"strict-transport-security"=>["max-age=15768000"]}
*** /validateLogin request headers ***
{"accept-encoding"=>["gzip, deflate, br"],
"accept"=>["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],
"user-agent"=>
["Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0"],
"content-type"=>["application/x-www-form-urlencoded"],
"accept-language"=>["en-US,en;q=0.9"],
"cache-control"=>["max-age=0"],
"connection"=>["keep-alive", "keep-alive"],
"content-length"=>["115"],
"cookie"=>["JSESSIONID=5BEF2E080C765C10CA285812B29C1FD2.tomcat8"],
"host"=>["www.idrive.com"],
"origin"=>["https://www.idrive.com"],
"referer"=>["https://www.idrive.com/idrive/login/loginForm"],
"upgrade-insecure-requests"=>["1"],
"keep-alive"=>["30"]}
*** /validateLogin POST data
support=&csfrtoken=XmJhSdU3qBaWbxJrp6oBymeLXEZdHha2dTmrY2DV&email=MYUSERNAME&password=MYPASSWORD&from=singleLogin
*** /validateLogin response headers ***
{"server"=>["nginx"],
"date"=>["Wed, 18 Jul 2018 17:05:41 GMT"],
"content-length"=>["0"],
"connection"=>["keep-alive"],
"x-frame-options"=>["SAMEORIGIN", "SAMEORIGIN"],
"location"=>["https://www.idrive.com/idrive/logout"],
"x-xss-protection"=>["1; mode=block"],
"x-content-type-options"=>["nosniff"],
"strict-transport-security"=>["max-age=15768000"]}
I can usually figure these things out but I have tried with Net::HTTP, Net::HTTP::Persistent, persistent_http, and httpclient, all with the same results. There may be something I am missing here but I would imagine that if a browser can do it, so can a script.
Aucun commentaire:
Enregistrer un commentaire