http.js not working with marketwatch urls

Sample URL: http://www.marketwatch.com/news/story.asp?guid=%7B21005575-02D4-D4B5-4572-D28A72DE3461%7D&siteid=rss&rss=1

Produced 301 Location: https://www.marketwatch.com/news/story.asp?guid=%7B21005575-02D4-D4B5-4572-D28A72DE3461%7D&siteid=rss&rss=1

Which produced 301 location: /story/guid/21005575-02d4-d4b5-4572-d28a72de3461?siteid=rss&rss=1

Which produced 301 location: http://www.marketwatch.com/story/the-new-york-stock-exchange-president-says-the-stock-market-is-not-a-casino-heres-what-academic-research-says-11613396805

which produced 301 Location: https://www.marketwatch.com/story/the-new-york-stock-exchange-president-says-the-stock-market-is-not-a-casino-heres-what-academic-research-says-11613396805

Which finally gives me a 200.

The code I'm using is

    var req = new HTTPRequest();
    req.follow_redirects=10;
    response = req.Get(url);
    writeln("RC: "+req.response_code);
    writeln("RS: "+req.response_status);
    writeln(response);

and RC is giving me a 404 and can't figure why.

How are the URLs "producing" 301s if not via http.js? The description says it ends up with a 200, but then you say you get a 404... it's certainly not both.

When I run the script the output from the section above is:

Working on http://www.marketwatch.com/news/story.asp?guid=%7B0F3C2883-C89D-472A-9721-7859431504DF%7D&siteid=rss&rss=1 RC: 404 RS: undefined

So, you tell me. If I manually go through the redirects, I get to the right place. If I use my script that eventually uses http.js then I get a 404.

So how are you getting 301s? How are you "manually" going through the requests?

Using the browser debugger.

So, a bit of feedback... in the description, tell us what is happening. If you want to add colour, make it clear what you're actually doing.

Also, it would be nice if when you include a script it is complete, works, and shows the problem (ie: load http.js and sets the url).

closed with commit aeb26cba

mentioned in commit aeb26cba

mentioned in commit dfd7e11c

mentioned in commit 45fd834a

mentioned in commit ae6c104e