|
|
Post by naturalselection on Jan 9, 2016 15:35:36 GMT
Developed my own bot this afternoon after being convinced 100% there's bots operating on the platform - let's see how this pans out!
|
|
james
Posts: 2,205
Likes: 955
|
Post by james on Jan 9, 2016 15:40:03 GMT
...A bot is just "doing what a human does, but faster" - and that, in itself, is a giveaway. Any bot worth running will be easily identifiable from server logs - because it will be refreshing the available loans page constantly and rapidly, in a way no human can sustain. ... That isn't easy though. You need to first study the structure of the different bots you are up against, and construct some sort of classification algorithm which filters real users from (sometimes quite sophisticated) bots. You would need to have an essentially zero false positive rate, as you can't go around banning genuine users for accidentally operating in a way in which your algorithm determines is 'bot like'. I don't know of many humans that will by hand load more than say 100 purchase-related pages per day. So set a limit on fast responses per account then increase the response time with each request. Since strong ID checks are required it's not going to be possible to circumvent with multiple accounts. 100 too few? Say what non-automation need there is for more and a higher than 100 level can be set before the site starts to deliberately slow down. No need to ever slow down things like getting details of a loan's properties, since it's the transaction-related pages that are the issue. This sort of thing is dirt cheap for a web site to do when it comes to ongoing running cost. However, while it's cheap to operate, it's not actually addressing something closer the real problem, which is in part finding some way to help those who want to buy on the secondary market do that in some sort of sensible allocation setup. It doesn't matter how fast an automation tool is if it's beaten by the site itself allocating the money to those who want to buy before its' displayed to anyone.
|
|
|
|
Post by naturalselection on Jan 9, 2016 16:10:10 GMT
Developed my own bot this afternoon after being convinced 100% there's bots operating on the platform - let's see how this pans out! Well who will deny the existence of bots on the sm now! They've probably existed from the early days... Also made a nice projection tool (something that might be nice on the website maybe?), fake figure example: pastebin.com/jizBKeNd
|
|
|
|
Post by nanniema on Jan 9, 2016 16:16:23 GMT
As used by Virgin Media to periodically sign in to email. Can be a PITA but never get any robot generated spam.
|
|
|
|
Post by sunspot on Jan 9, 2016 16:45:41 GMT
A captcha system will still involve watching the website 24/7, so even though it may partially defeat bots, it doesn't really solve the problem. In fact, it may even raise frustration levels higher.
The only sensible solution involves queued prefunding - but when I suggested this (with a poll) you emphatically voted it down. Well, I'm afraid well-written bots would be very difficult to detect, so banning them isn't really practical at a technical level.
Thus, we are left with four choices...
1) The status quo.
2) Handicapping - i.e. putting recent buyers at a disadvantage, perhaps by freezing requests by up to 60 seconds.
3) Captchas. However, an attended bot would still be massively advantageous, since the operator would have only one thing to do.
4) Queued prefunding (as I outlined in another thread).
Handicapping could be made to work, but would only really be effective when many loan parts become available at once - and even then, it would only result in the spoils being shared between those users who happened to be logged on at the time.
Captchas are generally disliked (and I've never seen them operating anywhere where users are already logged in) therefore they are unlikely to do much for user satisfaction.
To me, queued prefunding is a no-brainer. Anyone who disagrees, clearly has too much time on their hands (or a working bot).
|
|
locutus
Member of DD Central
Posts: 1,059 
Likes: 1,622
|
Post by locutus on Jan 9, 2016 17:14:51 GMT
Without lenders there is no money for SS to fund deals. If lenders continue to struggle to lend their money out at SS, then they will find other homes for it. Look at Money Thing and how fast that is growing. Ed continues to search for new opportunities to please his lenders and that is attracting more lenders and higher investment. SS has said how much they want their business to grow and to achieve this aim they need a dedicated, satisfied core of lenders. Lenders are the golden goose because without us SS would need to revert back to the old Lendy model of using their own money to fund loans. Your argument suggests that you are using a bot on the sm. Is that the case? Funny. SS need lenders. They probably don't care if it's little people like us or it's big people with big pockets with plenty to spare to buy a bidding bot from rentacoder.com. I expect it'd cost less than, say, £500 to buy a pretty good one. I do my buying just after a big loan goes live, there's usually plenty available. I don't expect to be able to buy at any time just because I want to. £500? You could build a half decent one using something like the following macro recorder imacros.net/browser/fx/welcome(I don't use bots but may be a bot)
|
|
|
|
Post by naturalselection on Jan 9, 2016 18:19:31 GMT
In-browser scripting isn't going to compete with a concurrent bot like I've trialed today - you'd lose 99.99% of the time due to latency.
|
|
|
|
Post by sunspot on Jan 9, 2016 18:19:32 GMT
A recorded macro would not be nearly up to the job, for one thing, any "half-decent" bot would need to read and understand the size of the loan on offer, and act accordingly. It would also need an input box by which to enter a prefunding limit, and a means of adding up. Beyond that, it would also need to check the response (i.e. success or fail) and take measures to avoid detection.
Something like the software indicated might be up to the job, but it would still require coding, and experience of reverse-engineering in order to figure out what code was required. You would also need another bank account to conduct testing (unless you want to be caught). Overall, it's a non-trivial task, and any that are already operating were probably written by people with genuine experience, rather than by wannabes.
FAO: NaturalSelection
Any bot operating as you've described would be trivial to detect simply by inspecting server logs - and if SS were of a mind to do so, they could simply cash you out, and close you down.
|
|
ablender
Member of DD Central
Posts: 2,204
Likes: 555
|
Post by ablender on Jan 9, 2016 18:30:55 GMT
Developed my own bot this afternoon after being convinced 100% there's bots operating on the platform - let's see how this pans out! Really? And you are willing to share the source code with all of us I suppose?
|
|
|
|
Post by naturalselection on Jan 9, 2016 18:48:26 GMT
Depends, not sure I'd like to aggravate the issue, partially at my own expense by releasing the source code. I'm happy to demo it tho.
I think mine might be hard to detect. The new Saving Stream polls a JSON API every second when you have a browser open on the available loans - it does this for every active user!
Ultimately SS need to sort this out (there are ways, some have been mentioned).
|
|
littleoldlady
Member of DD Central
Running down all platforms due to age
Posts: 3,045
Likes: 1,862
|
Post by littleoldlady on Jan 9, 2016 18:50:39 GMT
A captcha system will still involve watching the website 24/7, so even though it may partially defeat bots, it doesn't really solve the problem. In fact, it may even raise frustration levels higher. The only sensible solution involves queued prefunding - but when I suggested this (with a poll) you emphatically voted it down. Well, I'm afraid well-written bots would be very difficult to detect, so banning them isn't really practical at a technical level. Thus, we are left with four choices... 1) The status quo. 2) Handicapping - i.e. putting recent buyers at a disadvantage, perhaps by freezing requests by up to 60 seconds. 3) Captchas. However, an attended bot would still be massively advantageous, since the operator would have only one thing to do. 4) Queued prefunding (as I outlined in another thread). Handicapping could be made to work, but would only really be effective when many loan parts become available at once - and even then, it would only result in the spoils being shared between those users who happened to be logged on at the time. Captchas are generally disliked (and I've never seen them operating anywhere where users are already logged in) therefore they are unlikely to do much for user satisfaction. To me, queued prefunding is a no-brainer. Anyone who disagrees, clearly has too much time on their hands (or a working bot). The number of people who voted against your idea was tiny even though it was a majority. However your idea generated some serious questions so rather than leave the thread in a strop why not revisit it and elaborate on how it would work. The thread was hijacked by AC investors but ignore them.
|
|
locutus
Member of DD Central
Posts: 1,059
Likes: 1,622
|
Post by locutus on Jan 9, 2016 19:16:35 GMT
Depends, not sure I'd like to aggravate the issue, partially at my own expense by releasing the source code. I'm happy to demo it tho. I think mine might be hard to detect. The new Saving Stream polls a JSON API every second when you have a browser open on the available loans - it does this for every active user! Ultimately SS need to sort this out (there are ways, some have been mentioned). I've just been playing with Selenium and reckon I could write the required functionality within a couple of hours (can test for desired loan and also desired amount available as well as automatically buy of course). What is your dedicated bot written in out of interest?
|
|
|
|
Post by naturalselection on Jan 9, 2016 19:28:15 GMT
Depends, not sure I'd like to aggravate the issue, partially at my own expense by releasing the source code. I'm happy to demo it tho. I think mine might be hard to detect. The new Saving Stream polls a JSON API every second when you have a browser open on the available loans - it does this for every active user! Ultimately SS need to sort this out (there are ways, some have been mentioned). I've just been playing with Selenium and reckon I could write the required functionality within a couple of hours (can test for desired loan and also desired amount available as well as automatically buy of course). What is your dedicated bot written in out of interest? I'm using Google Go / Golang, the concurrency / networking packages were practically made for this task!
|
|
mikes1531
Member of DD Central
Posts: 6,453
Likes: 2,320
|
Post by mikes1531 on Jan 9, 2016 23:41:49 GMT
I do my buying just after a big loan goes live, there's usually plenty available. I expect that there will be more bots in use the next time a new large loan goes live than there were the last time. So it will be interesting to see whether there still will be plenty of parts available after the next big loan is released for those of us not using bots, or whether the army of bots will hoover up everything in sight. With there having been a reasonable amount of time elapsed since the last big loan release, there's bound to be a fair amount of demand built up in the interim, so there will be lots of competition for any available parts. And if any overdue loans actually are repaid in the meantime, that would exacerbate the situation. I think I'm going to need to increase my pre-funding amounts.
|
|
|
|
Post by GSV3MIaC on Jan 10, 2016 9:36:57 GMT
Depends, not sure I'd like to aggravate the issue, partially at my own expense by releasing the source code. I'm happy to demo it tho. I think mine might be hard to detect. The new Saving Stream polls a JSON API every second when you have a browser open on the available loans - it does this for every active user! Ultimately SS need to sort this out (there are ways, some have been mentioned). Actually it is every =three= seconds, getting a .json object from "https://savingstream.co.uk/_data-feed/renderLoansIfAny:available", which it then uses to decide whether to bother to redraw the available loans page in full. The JSON object is only about 30 bytes, so it is very efficient, compared to FC's nonsense where they'd send you half a GB just to tell you your balance hadn't changed. As I said in some other thread (now lost) there definitely are bots on the SM, and I own one now called Buddy (derived from my FC bot, Biddy) .. however I still vote for some queue / allocation system, as fairer for all. fwiw mine is written in VBA, and uses excel spreadsheet to tell it how much of each loan I want to own (hence it'll buy / sell to suit) .. basically duplicating the AC functionality (with a few extra wrinkles). No, folks, you can't have the code, I don't need the maintenance work, competition, etc. Since there is no 'flip for a profit' the only real motivation for botting Savingstream is to avoid having to do all my shopping in a frantic hurry at 1 AM. And no, I'm not a big hitter looking for 100k chunks, so you won't even notice me ... Buddy has been told not to buy more than £x in one bite, where x is a small 3 digit number, and it won't mess with shrapnel either. 8>.
|
|
