Current loan availability tracker

Quick kludge in place... hopefully that should keep thing ticking over for a bit.

Hi Ben,
Welcome to this forum. Are you open for smaller investors as well?

Cheers,

hnwlending hello and welcome.

Im happy to try to add you to my little tracker. To do that I would need visibility of availability. I've had a look around but can't see it... care to point me in the right direction?

Isnt it the last column on the loans list? (need to have registered) Click on heading to bring to top.

Hey jonah - thanks for the tag!

Yes, you're right, storing unencrypted credentials in your script wouldn't be advisable.... but there are a few alternatives, some of which would require cooperation of the platforms:

1) Platform makes the information you require public on their website, and not behind a login
2) Platform makes the information semi-public - i.e. they make it available via a specific URL that whilst is publicly accessible, isn't publicised (i.e. not linked to via from their website, and excluded from indexing by search engines, etc)
3) Platform develops an API (I very long shot, I know!)

However, if a platform doesn't want to do any of those things, but they are still happy for you to scrape information from behind their login gateway, then depending upon how they're going about their "session cookies", you may be able to go down this avenue...

Ordinarily, when you successfully login to a site, the site returns a unique & random session "token", which is stored in your browser as a "session cookie" (which persists until you either logout or close your browser). Then instead of then having to re-send your username/password with each subsequent http request, your browser instead sends this unique session "token" along with the request, which the server then uses to identify your "authenticated" logged in session.

Therefore, you could potentially manually login to a platform from the server where your script is running, grab the returned session cookie, and then insert this token into your script, and send it with each http request your script makes.

As long as you're making fairly regular calls from your script (i.e. more frequently than the "idle timeout" that the site employs), you may theoretically be able to remain logged in indefinitely.

Now, there is a caveat to that; some platforms may also employ an absolute session timeout (i.e. regardless of whether you're "idle" or not, the server will automatically expire your session token after X hours/days/weeks), but I know that many platforms don't!

Assuming an absolute session timeout isn't in play; then all your script would contain would be a unique session token, and not your login credentials.

If a platform is doing its session tokens correctly, then they should only be valid for the specific IP address (and in some instances also only for the specific browser) they were initially assigned to.

This would mean that even if the source code for your "script" was revealed in some way, exposing the session token you'd been using, it would be of no use to anything outside of your server. (i.e. if someone knew your session token and tried to make a request from their own server/browser using it (known as "session hijacking"), the site would simply reject it)

Of course, the above is quite a long-winded & tedious work-around; its reliability would very much depend on whether the platform allows users to remain logged in "indefinitely" or not, and its security would very much depend on whether the platform prevents "session hijacking" ("session hijacking" wasn't one of the factors I covered in my recent research, primarily because I don't have accounts with ALL 39 platforms I looked at in order to be able to fairly analyze this)

The best solution - if a platform really wants to appear in your tracker - would be for them to just make the necessary information "publicly" available for you to scrape.

Hope that helps!

I was thinking about if there is a better way to display this information, hence the demo below.

Please note this is a single snapshot for each time period, which is taken once an hour, just after the hour. If you are looking at a fast moving secondary market, it is possible that this snapshot could be unrepresentative of the time just before or just afterwards. It is however the value provided by the site at the time my script downloaded it.

Yeah, right.