I need to mirror a whole website asap - MyLifeMyID

Dr Lave · **Joined:** 30th Mar, 2008 **Posts:** 5979

Peter Perpendicularly (J Nash) just wrote a post over on WoS (reproduced below) about how MyLifeMyID has burned through it's £76,000 and is closing today/tomorrow.

It being somewhat of a testimant to how intelligent witty and erudite young people are at disagreeing with the government I really agree that it needs a full copy being made.

I've been fiddling with wget but I've only ever used it once on a very simple site so this complex forum one is confusing me a little.

So can any of you wonderful IT bods tell me help me mirror the whole kit and kaboodle?

It will be interesting to have a copy of the site once the official report tries to spin support for ID cards out of an unmitigated disaster. .

Quote:

As you may know, the Oy Vey! My Life! site tragically closes down tomorrow (Wednesday 15th) because the Home Office's grant of taxpayers' money to Virtual Surveys Inc, £76,249, runs out.

I didn't because like 103% of their readers I haven't been there for months, so therefore spent the ENTIRE DAY snapping every single page of the popular and influential debate hotbed* in order that the facts of the wretched enterprise can be compared to the final report when Jacqui Smith decides it won't be coming out after all.

The hideous layout of the grotesque forum software set up by Virtual Surveys Inc's crack team of incompetence-paralysed script-regurgitating government employees means I didn't actually read any of the pages because all of my eyes were too busy at sick, but a few bits did pop out as they slithered past in a concussing haze. The head of Virtual Surveys Inc gamely defending the £76,249 contract to bothersomely inquisitive taxpayers, for example. I didn't spot it, but hope the tremendously funny sequence still exists which involved a group of extravagantly authentic The Youths turning up and saying, “You all sound like my dad. I'm supporting ID cards out of spite. Zing!” It was like a 3fps goonrush.

If you hurry, you might still be in time to sign up and leave a poignant message in the goodbye thread ("Final Plea," current posts: one), or at least see if the pages still crash if you leave off a closing style tag. Or use the time instead speculating on how often Jacqui Smith thought about The Youths after spending £76,249 to secure a couple of minutes of news noddies on launch day. The answer is once, when she thought a passer-by was going to stab her because the sun had gone down, but she quickly passed a law so everything was all right.

*After not using my browser's built-in archiver for some reason, ie I didn't realise there was one. Blah.

Grim... · **Posted:** Wed Oct 15, 2008 16:00

Nightmare. Um - you're going to struggle.

Grim... · **Posted:** Wed Oct 15, 2008 16:01

What about this?
http://www.surfoffline.com/
I've not used it, so use at your own risk, etc.

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32619

Code:

wget -e robots=off -m http://www.mylifemyid.org

should do it., Seems to be working OK for me.

MrD · **Joined:** 30th Mar, 2008 **Posts:** 2139

Why don't you ask JN what his 'browser's built in archiver' is?
Maybe there's a mode on it where it simply slurps up each page as you go to it.

I would probably just use wget and tell it to use it's recursive web slurp.

Code:

wget -m -p -np (whatever the link to the top location of the forum is)

You may need to figure out some way to specify your cookies if you need to be logged in to read the forum at all.

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32619

Yeah, wget hasn't co-operated with drupal at all. Can't someone ask the guy to post a SQL dump?

Dudley · **Joined:** 30th Mar, 2008 **Posts:** 8679

If you've failed when I get home I'll dig out whatever I used to take a sneaky mirror of World of ("Linda Barker" - ed)

Dr Lave · **Joined:** 30th Mar, 2008 **Posts:** 5979

I don't have a windows PC for that Grim... cheers though.

I'm trying richardgaywood's wget fu at present. Thanks.

I don't think you need to log in to read, but when you click next page of comments in loads within the same url.

Getting that is what worries me.

I forgot about the site, and I don't really ahve time to visit and save each page. I'm going to try and grab some juicy ones tonight, but a full backup would be excellent.

Dr Lave · **Joined:** 30th Mar, 2008 **Posts:** 5979

Dudley wrote:

If you've failed when I get home I'll dig out whatever I used to take a sneaky mirror of World of ("Linda Barker" - ed)

That would be excellent if you could Duds.

I really think there is going to be a report that says things like "On the whole there was support" that can then be demolished with a big of grind through a copy of the site.

Kern · **Posted:** Wed Oct 15, 2008 16:13

And, as ever, East Germany's Jacqui Smith* won't care. I recently wrote to my MP for the umpteenth time on this subject. Rather than rehash the whole thing yet again I just pointed out that as the government is unable to answer the basic question of 'what problem are you trying to solve', is unable or unwilling to sensibly reply to criticism, and displays contempt for the public by being misleading about its own legislation. I then asked why nobody appears to have noticed the lack of clothing on the emperor. It would have been more productive to send him a turd in a jar given the trite reply I received.

* A tad unfair. At least East Germany kind of worked. We're getting the Railtrack version of it.

Dudley · **Joined:** 30th Mar, 2008 **Posts:** 8679

Ah you don't have Windows.

I'll have to do it myself since whatever I used was windows.

MarzSyndrome · **Joined:** 1st Apr, 2008 **Posts:** 426

You could try Offline Explorer Enterprise - though you would need to apply a crack to unlock its full potential. Contact me if you're interested.

EDIT: Except you just said you have no Windows PC. Urk. I guess the offer can still apply to someone else though.

Dr Lave · **Joined:** 30th Mar, 2008 **Posts:** 5979

richardgaywood wrote:

Yeah, wget hasn't co-operated with drupal at all. Can't someone ask the guy to post a SQL dump?

I don't know what you mean by drupal, but my version seems to have not worked either.

Ask the MyLifeMyID people? I'm doubting they will help.

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32619

Lave wrote:

I don't know what you mean by drupal, but my version seems to have not worked either.

Drupal is their CMS. The "/node/" in the URL is a giveaway.

Quote:

Ask the MyLifeMyID people? I'm doubting they will help.

Yeah I see that now, I didn't get what the site was when I posted that.

Curiosity · **Posted:** Wed Oct 15, 2008 16:20

I have a log-in to that site, after the last time you linked to it. I also complained to my MP about it too, though the response I got was notably only for its absence.

Dr Lave · **Joined:** 30th Mar, 2008 **Posts:** 5979

MrD wrote:

Why don't you ask JN what his 'browser's built in archiver' is?
Maybe there's a mode on it where it simply slurps up each page as you go to it.

I would probably just use wget and tell it to use it's recursive web slurp.

Code:

wget -m -p -np (whatever the link to the top location of the forum is)

You may need to figure out some way to specify your cookies if you need to be logged in to read the forum at all.

Gah, that doesn't work either. Thanks though.

Be Excellent To Each Other

I need to mirror a whole website asap - MyLifeMyID
Help me internet superstars!

Who is online

Be Excellent To Each Other

I need to mirror a whole website asap - MyLifeMyID Help me internet superstars!

Who is online

I need to mirror a whole website asap - MyLifeMyID
Help me internet superstars!