can i use web crawler to find a specific information from thousands of webpages

i am not a web expert less web coding but more of a low level developer. i am considering the web portal online database startup and one of the pressing issue is can i use the web crawler to pull specific information intelligently from a thousands of web page: examples are: building a database of 10-Q, 10-k report of thousands of companies. So web crawler visits each company page and find that information and pulls it and integrates into database.

I am not asking for specific code or examples, so question is whether this is possibility at all using web crawler or something else. If so, what are the approach? Any ready library, plug-in? language to be used? Thanks!

Insert a javascript inside infoview Google Maps api

I was trying to learn google maps api. I read about the infoview, and tried to insert a script inside content of the infoview, like below:

var text="<form action='www.google.com' method ='get'>"+
            "<input type='image' src="+b+"  width='48' height='48'>"+
            "</form>"+
            "<h1>"+a+"</h1>"+ 
            "<script type='text/javascript'>alert('It's okay!')</script>";

but I get this error:

Uncaught SyntaxError: Invalid or unexpected token

on the line of the script, and when i try to execute the page, the browser show me the html code.

Could someone explain me why it happen ?

"Web Development with Clojure": No such var: db/get-messages

I am working through the Luminous "Web Development With Clojure 2nd Edition" book and have an issue. I checked the home.clj to make sure the aliased db is set as guestbook.db.core.

When I compile my code, I receive:

Exception in thread "main" java.lang.ExceptionInInitializerError
at clojure.main.<clinit>(main.java:20)

Caused by: java.lang.RuntimeException: No such var: db/get-messages, compiling:(guestbook/routes/home.clj:16:22)
at clojure.lang.Compiler.analyze(Compiler.java:6688)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3766)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6870)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler$MapExpr.parse(Compiler.java:3072)
at clojure.lang.Compiler.analyze(Compiler.java:6677)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3834)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6870)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3834)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6870)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001)
at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6319)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6856)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5380)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3972)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6866)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6856)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.access$300(Compiler.java:38)
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:589)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868)
at clojure.lang.Compiler.analyze(Compiler.java:6669)
at clojure.lang.Compiler.analyze(Compiler.java:6625)
at clojure.lang.Compiler.eval(Compiler.java:6931)
at clojure.lang.Compiler.load(Compiler.java:7379)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.loadResourceScript(RT.java:363)
at clojure.lang.RT.load(RT.java:453)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__5677.invoke(core.clj:5893)
at clojure.core$load.invokeStatic(core.clj:5892)
at clojure.core$load.doInvoke(core.clj:5876)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5697)
at clojure.core$load_one.invoke(core.clj:5692)
at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
at clojure.core$load_lib.invokeStatic(core.clj:5736)
at clojure.core$load_lib.doInvoke(core.clj:5717)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$load_libs.invokeStatic(core.clj:5774)
at clojure.core$load_libs.doInvoke(core.clj:5758)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$require.invokeStatic(core.clj:5796)
at clojure.core$require.doInvoke(core.clj:5796)
at clojure.lang.RestFn.invoke(RestFn.java:551)
at guestbook.handler$eval218$loading__5569__auto____219.invoke(handler.clj:1)
at guestbook.handler$eval218.invokeStatic(handler.clj:1)
at guestbook.handler$eval218.invoke(handler.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6927)
at clojure.lang.Compiler.eval(Compiler.java:6916)
at clojure.lang.Compiler.load(Compiler.java:7379)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.loadResourceScript(RT.java:363)
at clojure.lang.RT.load(RT.java:453)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__5677.invoke(core.clj:5893)
at clojure.core$load.invokeStatic(core.clj:5892)
at clojure.core$load.doInvoke(core.clj:5876)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5697)
at clojure.core$load_one.invoke(core.clj:5692)
at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
at clojure.core$load_lib.invokeStatic(core.clj:5736)
at clojure.core$load_lib.doInvoke(core.clj:5717)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$load_libs.invokeStatic(core.clj:5774)
at clojure.core$load_libs.doInvoke(core.clj:5758)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$require.invokeStatic(core.clj:5796)
at clojure.core$require.doInvoke(core.clj:5796)
at clojure.lang.RestFn.invoke(RestFn.java:619)
at guestbook.core$eval212$loading__5569__auto____213.invoke(core.clj:1)
at guestbook.core$eval212.invokeStatic(core.clj:1)
at guestbook.core$eval212.invoke(core.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6927)
at clojure.lang.Compiler.eval(Compiler.java:6916)
at clojure.lang.Compiler.load(Compiler.java:7379)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.loadResourceScript(RT.java:363)
at clojure.lang.RT.load(RT.java:453)
at clojure.lang.RT.load(RT.java:419)
at clojure.core$load$fn__5677.invoke(core.clj:5893)
at clojure.core$load.invokeStatic(core.clj:5892)
at clojure.core$load.doInvoke(core.clj:5876)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invokeStatic(core.clj:5697)
at clojure.core$load_one.invoke(core.clj:5692)
at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
at clojure.core$load_lib.invokeStatic(core.clj:5736)
at clojure.core$load_lib.doInvoke(core.clj:5717)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$load_libs.invokeStatic(core.clj:5774)
at clojure.core$load_libs.doInvoke(core.clj:5758)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invokeStatic(core.clj:648)
at clojure.core$require.invokeStatic(core.clj:5796)
at clojure.core$require.doInvoke(core.clj:5796)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at user$eval3$loading__5569__auto____4.invoke(user.clj:1)
at user$eval3.invokeStatic(user.clj:1)
at user$eval3.invoke(user.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6927)
at clojure.lang.Compiler.eval(Compiler.java:6916)
at clojure.lang.Compiler.load(Compiler.java:7379)
at clojure.lang.RT.loadResourceScript(RT.java:372)
at clojure.lang.RT.loadResourceScript(RT.java:359)
at clojure.lang.RT.maybeLoadResourceScript(RT.java:355)
at clojure.lang.RT.doInit(RT.java:475)
at clojure.lang.RT.<clinit>(RT.java:331)
... 1 more

Caused by: java.lang.RuntimeException: No such var: db/get-messages
at clojure.lang.Util.runtimeException(Util.java:221)
at clojure.lang.Compiler.resolveIn(Compiler.java:7138)
at clojure.lang.Compiler.resolve(Compiler.java:7108)
at clojure.lang.Compiler.analyzeSymbol(Compiler.java:7069)
at clojure.lang.Compiler.analyze(Compiler.java:6648)
... 127 more

Here's my project.clj file

(defproject guestbook "0.1.0-SNAPSHOT"



:description "FIXME: write description"
  :url "http://ift.tt/JpfjOb"

  :dependencies [[org.clojure/clojure "1.8.0"]
         [selmer "1.0.7"]
                 [markdown-clj "0.9.89"]
                 [ring-middleware-format "0.7.0"]
                 [metosin/ring-http-response "0.8.0"]
                 [bouncer "1.0.0"]
                 [org.webjars/bootstrap "4.0.0-alpha.3"]
                 [org.webjars/font-awesome "4.6.3"]
                 [org.webjars.bower/tether "1.3.3"]
                 [org.webjars/jquery "3.0.0"]
                 [org.clojure/tools.logging "0.3.1"]
                 [compojure "1.5.1"]
                 [ring-webjars "0.1.1"]
                 [ring/ring-defaults "0.2.1"]
                 [mount "0.1.10"]
                 [cprop "0.1.9"]
                 [org.clojure/tools.cli "0.3.5"]
                 [luminus-nrepl "0.1.4"]
                 [luminus-migrations "0.2.6"]
                 [conman "0.6.0"]
                 [com.h2database/h2 "1.4.192"]
                 [org.webjars/webjars-locator-jboss-vfs "0.1.0"]
                 [luminus-immutant "0.2.2"]]

  :min-lein-version "2.0.0"

  :jvm-opts ["-server" "-Dconf=.lein-env"]
  :source-paths ["src/clj"]
  :resource-paths ["resources"]
  :target-path "target/%s/"
  :main guestbook.core
  :migratus {:store :database :db ~(get (System/getenv) "DATABASE_URL")}



    :plugins [[lein-cprop "1.0.1"]
            [migratus-lein "0.4.1"]
            [lein-immutant "2.1.0"]]

  :profiles
  {:uberjar {:omit-source true
             :aot :all
             :uberjar-name "guestbook.jar"
             :source-paths ["env/prod/clj"]
             :resource-paths ["env/prod/resources"]}

   :dev           [:project/dev :profiles/dev]
   :test          [:project/test :profiles/test]

   :project/dev  {:dependencies [[prone "1.1.1"]
                                 [ring/ring-mock "0.3.0"]
                                 [ring/ring-devel "1.5.0"]
                                 [pjstadig/humane-test-output "0.8.1"]]
                  :plugins      [[com.jakemccrary/lein-test-refresh "0.14.0"]]`

                  :source-paths ["env/dev/clj" "test/clj"]
                  :resource-paths ["env/dev/resources"]
                  :repl-options {:init-ns user}
                  :injections [(require 'pjstadig.humane-test-output)
                               (pjstadig.humane-test-output/activate!)]}
   :project/test {:resource-paths ["env/dev/resources" "env/test/resources"]}
   :profiles/dev {}
   :profiles/test {}})`

home.clj file

(ns guestbook.routes.home
  (:require [guestbook.layout :as layout]
            [compojure.core :refer [defroutes GET]]
            [compojure.core :refer [defroutes GET POST]]
            [ring.util.http-response :as response]
            [clojure.java.io :as io]

            [guestbook.db.core :as db]
            [bouncer.core :as b]
            [bouncer.validators :as v]
            [ring.util.response :refer [redirect]]))

(defn home-page [{:keys [flash]}]
  (layout/render
   "home.html"
   (merge {:messages (db/get-messages)}
          (select-keys flash [:name :message :errors]))))

(defn about-page []
  (layout/render "about.html"))

(defroutes home-routes
  (GET "/" request (home-page request))
  (POST "/" request (save-message! request))
  (GET "/about" [] (about-page)))

(defn validate-message [params]
  (first
    (b/validate
      params
      :name v/required
      :message [v/required [v/min-count 10]])))

(defn save-message! [{:keys [params]}]
  (if-let [errors (validate-message params)]
    (-> (response/found "/")
        (assoc :flash (assoc params :errors errors)))
    (do
      (db/save-message!
       (assoc params :timestamp (java.util.Date.)))
      (response/found "/"))))

Database file

(ns guestbook.db.core
  (:require
    [conman.core :as conman]
    [mount.core :refer [defstate]]
    [guestbook.config :refer [env]]))

(defstate ^:dynamic *db*
           :start (conman/connect! {:jdbc-url (env :database-url)})
           :stop (conman/disconnect! *db*))

(conman/bind-connection *db* "sql/queries.sql")

Handler file

(ns guestbook.handler
  (:require [compojure.core :refer [routes wrap-routes]]
            [guestbook.layout :refer [error-page]]
            [guestbook.routes.home :refer [home-routes]]
            [compojure.route :as route]
            [guestbook.env :refer [defaults]]
            [mount.core :as mount]
            [guestbook.middleware :as middleware]))

(mount/defstate init-app
                :start ((or (:init defaults) identity))
                :stop  ((or (:stop defaults) identity)))

(def app-routes
  (routes
    (-> #'home-routes
        (wrap-routes middleware/wrap-csrf)
        (wrap-routes middleware/wrap-formats))
    (route/not-found
      (:body
        (error-page {:status 404
                     :title "page not found"})))))


(defn app [] (middleware/wrap-base #'app-routes))

queries.sql database

-- :name create-user! :! :n
-- :doc creates a new user record
INSERT INTO guestbook
(name, message, timestamp)
VALUES (:name, :message, :timestamp)


-- :name get-user :? :*
-- :doc retrieve a user given the id.
SELECT * FROM guestbook

Can someone please help me figure out why I'm stuck with this No such var: db/get-messages error at line 16: column 24? I tried following this similar issue, but it didn't solve my problem ("Web Development with Clojure": No such var: db/get-user).

How to save request response in EVOThing application with a JS Script

This could potentially be a stupid question, but I've been working for two weeks on it with no success. I'm using an Arduino101 that connects via bluetooth to an EVOThings application on a phone. The application requests tweeter information from a Flask server hosted on Heroku and the idea is to relay the response to the Arduino through the bluetooth connection. However, although the response goes through successfully, I'm having trouble parsing the response on the client side to do further processing and send it to the Arduino101. Below is some of the code:

<script>
        // Application object.
        var app = {}
        // Connected device.
        app.device = null;
        app.msg = '';
        
        //Get the message.
        app.getMsg = function()
        {       
                $.ajax({ type: "GET",
                         contentType: "application/json; charset=utf-8",
                         url: 'http://ift.tt/2bmUWzE',   
                         async: false,
                         dataType: "json",
                         //navigator.notification.alert("hello", function() {});
                         success : function(text)
                         {
                                // navigator.notification.alert("hello", function() {});
                                 var res = JSON.stringify(text.data);
                                // navigator.notification.alert("hello", function() {});
                                 //document.getElementById("try").innerHTML=res;
                                 var message = '';
                                 for(i = 0; i < res.length; i++){
                                        message += res[i];
                                 }
                                 app.msg += message;
                                 //navigator.notification.alert("hello", function() {});
                                 
                         }
                });
        }
        // Turn on LED red.
        app.ledRed = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([1]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
        
        // Turn on LED green
        app.ledGreen = function()
        {
                navigator.notification.alert(app.msg, function() {});
                app.device && app.device.writeDataArray(new Uint8Array([2]), '19b10001-e8f2-537e-4f6c-d104768a1214');   
        }
                // Turn on LED blue
        app.ledBlue = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([3]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
                // Turn on LED yellow
        app.ledYellow = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([4]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
                // Turn on LED cyan
        app.ledCyan = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([5]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
                // Turn on LED purple
        app.ledPurple = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([6]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
                // Turn on LED white
        app.ledWhite = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([7]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
        
        // Turn off LED.
        app.ledOff = function()
        {
                app.device && app.device.writeDataArray(new Uint8Array([0]), '19b10001-e8f2-537e-4f6c-d104768a1214');
        }
        app.showMessage = function(info)
        {
                document.getElementById('info').innerHTML = info
        };
        // Called when BLE and other native functions are available.
        app.onDeviceReady = function()
        {
                app.showMessage('Touch the connect button to begin.');
        };
        app.connect = function()
        {
                evothings.arduinoble.close();
                app.showMessage('Connecting...');
                evothings.arduinoble.connect(
                        'ARDUINO 101-3E4E', // Advertised name of BLE device.
                        function(device)
                        {
                                app.device = device;
                                app.showMessage('Connected! Touch buttons to turn LED on/off.');
                        },
                        function(errorCode)
                        {
                                app.showMessage('Connect error: ' + errorCode + '.');
                        });
        };
        document.addEventListener(
                'deviceready',
                function() { evothings.scriptsLoaded(app.onDeviceReady) },
                false);
        </script>

What I did was modify the EVOThing example for connecting the Arduino101 to the phone app with BLE by adding a function to do the request using AJAX and saving it in a global variable, which I then plan to send to the Arduino. But the msg is never stored in the variable to begin with. Actually, even using the alert function to debug the success code block in the AJAX post, I get nothing. The server side is getting the response and is returning a OK 200 code as seen below: Heroku Logs from server side

For some reason the msg coming from the server is never saved to the app.msg variable. Any help on why this setup doesn't work will be greatly appreciated.

How can I view API requests made over my wireless network?

So I am wanting to see API requests made from my phone or other mobile devices that are made on my home network. I have downloaded Acrylic Wifi Professional but do not see how to view such data with it. Is there a better tool or way to see this data?

mern.io: server.js Unexpected token

I'm trying to build a simple app using mernio:

mern init myApp
cd myApp
npm install
npm start

I got the following error:

/Users/hazemhagrass/workspace/nodejs/vi_mern_server/node_modules/babel-core/lib/transformation/file/index.js:573 throw err; ^

SyntaxError: /Users/hazemhagrass/workspace/nodejs/vi_mern_server/server/server.js: Unexpected token (125:10) 123 | .then(() => { 124 |
const initialView = renderToString(

125 | | ^ 126 | 127 | 128 |

I Just Need to Have A PHP page run on my server - Not send it info or Get Info from it

Good Day;

I understand WWW and Php enough to have created a successful login and registration form, but all I want to do now is have my unity c# script to erase (delete) all of the data in a particular table. When I run the php page from my browser then check my database the records were deleted. But when I use the code I created below to (I hoped) only make the same page run, nothing gets deleted!

    void  OnServerInitialized (){
         //Erase Exisiting User Data from the database
         Debug.Log("In OnServerInitialized");
         WWW w = new WWW("http://ift.tt/2bTIQzz");
         StartCoroutine(ErasePlayerNames(w));
    }

    IEnumerator ErasePlayerNames(WWW w){
        yield return w;
    }

Note: I did change the ipaddress for this post for obvious reasons.

Please Inform me of what I am doing wrong, thanks.