How the Surly Survey works

You might have noticed the ad-like questions worded to get you to click on them, in the asides and the right spine of the website.

That is so you will click on them. Those are from my Surly Survey which is quite safe unless you don’t want me to know the answer to those questions. I do capture the results and show back to you everyone else’s aggregate answer. This is an early attempt to try something called lazy registration, though it works better as a survey and marketing tool. The idea is to make it easy for a reader, to enter information. More he reads, tempt him to enter more information, so you know more about him. And when he does register, just pre-populate the registration fields with what he has already entered.

The application is simple to write, though I suspect creating the content takes longer.

Requirements for inserting it in your webpage

  1. Requires you to be able to paste javascript in the page where you want the Surly Survey to appear. This isn’t always possible.
  2. The users must use modern browsers with XmlHttpRequest, with CORS (Cross Origin Resource Sharing) protocol. Almost all already do.

To implement the system that serves the content and responds to the answers

  1. You need a language capable of sending HTTP headers esp these in figure below. You can check if you can send these from your Web hosting provider.
    Access-Control-Allow-Origin: http://domain.of.webpage
    Access-Control-Allow-Credentials: true
  2. A web hosting service that will allow you to send the headers
  3. Knowledge of javascript and XmlHttpRequest API
  4. A database or file you can parse, to pull responses from

The central technology that makes this work, is built into most modern browsers

XmlHttpRequest has been included in browsers for over 20years now. The security protocol it implements to contain uses of malicious code is called CORS (Cross Origin Resource Sharing). To just make a simple GET request on a domain you didn’t get the webpage from, you can include this header on the response back to XmlHttpRequest.

Access-Control-Allow-Origin: *

It means the server is ok with XmlHttpRequest requesting data from it. If you do not enable this on the response, a modern browser will assume that it is executing a script that is bothering a server without permission and it will throw away the response, keeping the script from being able to use it.

Scenario Request Response
Make stateless cross-origin request HTTP Header (irrelevant) Access-Control-Allow-Origin: *
Responsiblity In webpage executed in browser, make regular XmlHttpRequest thru Javascript
var xhr = new XMLHttpRequest();
xhr.open('get', 'http://url', true);
xhr.send(null);
The software has to return the specific HTTP header, either thru web server or the program inserts it in response. Web Hosting providers do have the ability block certain headers, so you may need to confirm that it is supported by the hosting provider.
If you don't follow the step There is nothing special here about the request. The request will always get sent to server. The browser doesn't prevent it. Server will still send the response. The server isn't sure it's a XmlHttpRequest. But if Access-Control-Allow-Origin: * isn't sent as part of response, the XmlHttpRequest is supposed to throw the response away.
Make stateFUL cross-origin request HTTP Header Cookie: [XmlHttpRequest url's cookies] Access-Control-Allow-Origin: http://referrer domain
Access-Control-Allow-Credentials: true
Responsiblity You have to set .withCredentials property to true before send()
var xhr = new XMLHttpRequest();
xhr.open('get', 'http://url', true);
xhr.withCredentials = true;
xhr.send(null);
The software has to return the specific HTTP header, either thru web server or the program inserts it in response. Web Hosting providers do have the ability block, so you may need to confirm that it is supported there.
If you don't follow the step If you don't include the .withCredentials, XmlHttpRequest will not send cookies in the request. Obviously the server will process the request as if it had no cookies. It will still send the response. The server can't be sure it is xmlhttprequest. If it doesn't send both headers in the exact format, the browser is supposed to throw the away the response, because it is the only object that is sure it sent .withCredentials = true, it can know how to validate with which scenario listed here. This can be confusing because you will see the response in-flight on a network sniffer, but the browser will not show you the response results.
If you want to do Cross-Origin Requests from XmlHttpRequest

Above is the most distilled explanation I can think of. If you see code on other sites about instantiating different objects, or checking if property exists, it's just checking for compatibility with a standardized API.

BUT THE SCENARIO IN WHICH A SINGLE HTTP HEADER IS RETURNED IN THE RESPONSE IS INSUFFICIENT FOR THE SURLY SURVEY. IT NEEDS BOTH. Because the Surly Survey captures session state. When both headers are being sent in the response headers, the server is granting permission to XmlHttpRequest on the browser to use the data it sends (if you are XmlHttpRequest), and through Access-Control-Allow-Credentials : true, that it has acknowledged receiving cookies for XmlHttpRequest (if it is XmlHttpRequest and if cookies were indeed sent), too. I just added it to my Web Server's configuration to always output the same headers, rather than add them in the application code. Though, if you might want to try adding it to your application's HTTP response, in case you don't have ability to change the Web Server's configuration.

! Cannot use '*' when sending cookies

This combination of HTTP headers sent from the server might cause the browser to throw away the response, regardless of whether it sent cookies or not.

Access-Control-Allow-Origin: *X
Access-Control-Allow-Credentials: true

So we going to create the system described below. The green represents HTML on a page, served to a web browser. The highlighted code in first panel, represents code you paste on the server, before it was served to the browser.




You wouldn't think it was so complicated, would you?

The reference code is below...

Insert this <script> tag into a webpage, so you don't have to write the code in "fillTargetWithURL" over and over again. "fillTargetWithURL" can be any Web technology (PHP, ASP.NET) you wish, as long as it can accept a querystring and change the javascript in the content. "urlwithcontent" can be any url that contains the HTML content you wish displayed. And "targetElement" is the DOM element, you wish to replace with the contents of urlwithcontent. Ie. http://mywebsite/fillTargetWithURL.js?url=http://mywebsite/content&targetelementid=putitinthiselement.

<div id="targetElement">   
    Will get overwritten
</div>
<script src="fillTargetWithURL?url=[urlwithcontent]&targetelementid=targetElement">
</script>

fillTargetWithURL
Can be any web technology that produces this output, except with targetElement and urlwithcontent updated. A static HTML page is possible, but then it always points to same url and the target DOM object id has to be always the same. The output was requested from <script src>, so the response should be in Javascript. The javascript uses XmlHttpRequest to get "Real HTML Content" from urlwithcontent.


//bootstrap for inserting inline content from another url
var x = new XMLHttpRequest();
x.onload = function () {
    document.getElementById('targetElement').innerHTML = this.response;
}
x.open('get', 'urlwithcontent', true);
x.withCredentials = true;
x.send();

http://url/withcontent
This is the first page requested by XmlHttpRequest. Up until now, the server didn't need to return the HTTP Headers "Access-Control-Allow-Credentials: true" or "Access-Control-Allow-Origin:http://referrer domain" yet. But now in the response to the XmlHttpRequest, it needs to include those headers. We want to send cookies, so the "...origin" has the webpage's domain and "...access control" is set to true.

urlwithcontent again can be any technology that serves up dynamic content. It will read the cookie, or querystring values if submitted, and render HTML to be sent back to XmlHttpRequest. In this case, it renders a <form action="urlwithnewcontent"> and this means the form data is intended to be sent as a request to urlwithnewcontent. The inline javascript in the onsubmit event handler will intercept the submit before it is sent, and try to execute the form submission thru XmlHttpRequest. If successful, it will return a false to the form's event handler, to abort submitting to a target window(newwindow). We want the content urlwithcontent responded with, to match the HTML code below.

This should produce a form with 2 buttons, which will try to submit the user's answer thru XmlHttpRequest, and the response will be used to overwrite the form's original question.

<!--The entire form is wrapped with a div, so the form with questions can be overwritten on submit-->
<div id='targetOverwritten'>
    <!--You cannot create javascript functions thru InnerHTML, so you have to have inline functions-->
    <!--The target is to create a new window for submission if javascript fails-->
    <!--if javascript for XmlHttpRequest submission fails at any point, it returns true to enable submission by browser-->
    <form method="GET" target="newwindow" action="urlwithnewcontent" 
          onsubmit="var fm=this; 
                    if (!fm.action) return true;
                    var req = new XMLHttpRequest();
                    if (!req) return true; 
                    var target=document.getElementById('targetOverwritten');
                    if (!target) return true;  
                    req.onload = function() {
                        var target2=document.getElementById('targetOverwritten');
                        target2.innerHTML = this.response; 
                    } 
                    var url=fm.action+'?value=' + escape(fm.submitvalue); 
                    req.open('get', url, true); 
                    req.withCredentials = true;
                    req.send(null); 
                    return false; ">
                    [Snarky question]
    <!-- <input type=button> the displayed value is same as the submitted value, so we use <button> -->
    <button type='submit' name='value' value='True' onclick='this.form.submitvalue=this.value;'>Yes</button>
    <button type='submit' name='value' value='False' onclick='this.form.submitvalue=this.value;'>No</button>
    </form>
</div>

http://url/withnewcontent
This should also be a page built in the web tachnology of your choice. It was intended to receive data via XmlHttpRequest thru cross-site scripting. So it also needs the return the aforementioned HTTP headers.

But it's purpose, is to process the cookies, and the data in the querystring, and save it to the database. That code is not shown here. The output below, is what the script returns to the XmlHttpRequest object (hopefully) after the data has been saved

<h1>Thank you for your response</h1>

Eventually the code will be posted in Github as a small ASP.NET MVC project, and you can adjust to your purpose as needed.


See also, for IIS, it doesn't need to be application added:

Any origin:
https://enable-cors.org/server_iis6.html

For IIS7, specified origins:
https://www.iis.net/downloads/microsoft/iis-cors-module
https://blogs.iis.net/iisteam/getting-started-with-the-iis-cors-module

Both methods work for the described use case.

Leave a Reply

Your email address will not be published. Required fields are marked *