Some of AVM’s Fritz!Box routers allow connecting a USB device and use this device as a network attached storage (NAS) via Samba/SMB in the local network. In combination with the fact that the NAS can also be accessed from the Internet via FTP, and the multimedia files can be streamed to a TV via the Fritz!Mediaserver (using DLNA), it makes a pretty basic home entertainment system. I use it to automatically copy videos (YouTube, etc.) to the NAS from my virtual server, and then watch these with my Samsung Smart TV. It’s not as great as XBMC, but it works for now.
Unfortunately, the Fritz!Mediaserver (DLNA server) does not automatically refresh the index when media files are added via Samba/SMB or FTP (only if they are added via their web interface Fritz!NAS). It can be refreshed manually via the interface on fritz.box/storage/settings.lua. But, since I like to automate things, I made a little helper to automatically refresh the index.
Only once: Download and configure the Script
Setting up the script is easy. Just download and configure the Fritz!Box DLNA index refresh script. If you haven’t already, install the PHP command line interpreter:
|
1 2 3 4 5 |
sudo apt-get install php5-cli wget http://blog.philippheckel.com/uploads/2012/12/fritzbox-md5-2.1-BROKEN.js.php wget http://blog.philippheckel.com/uploads/2012/12/fritzbox-dlna-refresh chmod +x fritzbox-dlna-refresh vi fritzbox-dlna-refresh |
Now update the configuration (esp. your Fritz!Box router password) and run the script.
Side note (not relevant for the setup): If you’re wondering why an external MD5 library is necessary to run this script, here’s the answer. The Fritz!Box 6360 uses Paul Johnston’s MD5 JavaScript library (version 2.1). Unfortunately, this version of the library is broken and the MD5 hash it returns is wrong — in other words: md5(“123”) != hex_md5(“123”). So I had to translate the library from JavaScript to PHP. The resulting library can be found is at fritzbox-md5-2.1-BROKEN.js.php.
Running the script
The output of the script should look something like this:
|
1 2 3 4 |
./fritzbox-dlna-refresh [GET login.lua ] Retrieving login page and challenge ... challenge is 63dbb3af, calculated response is 63dbb3af-e8d4c113f2b5a73c80d5df7488d7bd84 [POST login.lua ] Logging in ... Successful, SID is ece4927d0b7b58c6 [POST settings.lua ] Refreshing NAS DB 1 ... Done. |
Of course this script also needs to be manually triggered somehow. But setting up a cronjob on you laptop or home server shouldn’t be a problem…
I hope this helped a little!
Hi Phillip,
I found this great script to mitigate the shortcoming of the fritzbox to update the media index just after new files are being uploaden. I wanted to use this script with my fritzbox 7360. I manages to get the script running but I get the following message:
php fritz_mediaserver_update.php
[GET login.lua ] Retrieving login page and challenge … challenge is be8839f2, calculated response is be8839f2-77772e1284fa0dcf268d01f8c1a77979
[POST login.lua ] Logging in … postlogin_response does not match.
Can you please help why this is not working?
Thanks Ron.
Forget about the typos -;)
I don’t have that Fritz!Box, so I can’t really try it out, but I’m guessing that the hash is built differently — or that the MD5 library is actually working properly.
Hi Philip,
Hope this is not the end of this -;) how to find out what MD5 library is used in my fritzbox? in other words, how did you find out?
Ron.
Hi Philip,
Had some second thoughts on this. If I understand this correctly your script first needs to logon on th fritz box. In order to do so, the script calculates the MD5 hash response based on the password configured in the script and some key(s). The response is submitted in order to logon to the fritz. What I’m not sure about is, if this key is only the challenge response from the fritz, or that you also use an additional key for calculating the hash response. Reason for asking is a script I found on http://svn.freetz.org/trunk/Config.in. If I look at this script it seems this includes different MD5 checksum values and are different for each fritzbox type. So my question is:
When calculating the MD5 hash response, are the only attributes needed to calculate this the challenge returned when logon AND the configured password in the script?
(reason for the confusion is that for instance PKI requires use of private/public key pairs)
HOW THE LOGIN WORKS
So I can’t really tell you how all the other Fritz!Boxes do the login, but the Fritz!Box 6360 login works based on a calculation of a MD5 sum. This is how my script works:
1. Retrieve Fritz!Box login page at http://www.fritz.box/login.lua
2. Parse the page and read the “challenge” (= short 8-byte checksum), like this: [“security:status/challenge”] = “894c809a”
3. When you click Login, the JavaScript function uiDoOnMainFormSubmit() concatenates the security challenge with the password you entered, like this:
var sec_resp = g_sec_challenge + “-” + dot_pass;
jxl.setValue(“uiResp”, g_sec_challenge + “-” + hex_md5(sec_resp));
dot_pass is the password you entered with all the non-ASCII characters dotted out (so “.berall” instead of “Überall”). g_sec_challenge is the 8-byte challenge you previously retrieved.
MD5 LIBRARY
Regarding the MD5 library, just grab any properly working JavaScript MD5 library you can find. In the newest version, this one works fine: http://pajhome.org.uk/crypt/md5/ — Maybe the Fritz!Box you have has a properly functioning one, maybe it hasn’t. I actually checked the library version at http://www.fritz.box/js/md5.js, then took the script and ported it to PHP. A lot of effort I tell you, only because this library was broken. Otherwise, the login script would have been really simple. Porting the faulty library took the longest.
TOOL SUPPORT
To find out how the login mechanism works with your Fritz!Box, install Firebug and TamperData to your Firefox, so you can monitor the network traffic and set breakpoints to see the calculated values.
Hope this help!
I still do not understand the Broken part, to be honest. Maybe a stupid question; can’t you calculate the hash response using a standard MD5 PHP function?. What is the hashing algoritm used in the java scripts you calculated?
The problem with the Fritz!Box 6360 is that AVM used a MD5 library that is broken — probably without knowing it — I just don’t know if other Firtz!Boxes use the same library.
The Fritz!Box 6360 uses this library http://pajhome.org.uk/crypt/md5 in version 2.1 (the broken version). If your version is the same (check the file here: http://www.fritz.box/js/md5.js), use the BROKEN_hex_md5()-function. If not, use the standard md5() function.
If you have the library in version 2.1: $response = $challenge.”-“.md5($response);
Otherwise: $response = $challenge.”-“.BROKEN_hex_md5($response);
I suppose the Fritz!Box uses hashing to prevent easy brute force attacks. Because you need the challenge to successfully login. Did that answer your question?
Hi Philip,
Thanks so far for the suggestions.
Checked the js library you mentioned on the box. Seems to be the same version then the one used on your box. Copied the md5.js from my box below. I also changed your script to use the standard md5 function; NO result. Man this is frustrating -;) Why is this not working!? Also checked the password configured in the script again; it is really the right one.
MD5.js rerieved from 7360
/*
* A JavaScript implementation of the RSA Data Security, Inc. MD5 Message
* Digest Algorithm, as defined in RFC 1321.
* Version 2.1 Copyright (C) Paul Johnston 1999 – 2002.
* Other contributors: Greg Holt, Andrew Kepert, Ydnar, Lostinet
* Distributed under the BSD License
* See http://pajhome.org.uk/crypt/md5 for more info.
*/
/*
* Configurable variables. You may need to tweak these to be compatible with
* the server-side, but the defaults work in most cases.
*/
var hexcase = 0; /* hex output format. 0 – lowercase; 1 – uppercase */
var b64pad = “”; /* base-64 pad character. “=” for strict RFC compliance */
/* var chrsz = 8; */ /* bits per input character. 8 – ASCII; 16 – Unicode */
var chrsz = 16; /* bits per input character. 8 – ASCII; 16 – Unicode */
/*
* These are the functions you’ll usually want to call
* They take string arguments and return either hex or base-64 encoded strings
*/
function hex_md5(s){ return binl2hex(core_md5(str2binl(s), s.length * chrsz));}
function b64_md5(s){ return binl2b64(core_md5(str2binl(s), s.length * chrsz));}
function str_md5(s){ return binl2str(core_md5(str2binl(s), s.length * chrsz));}
function hex_hmac_md5(key, data) { return binl2hex(core_hmac_md5(key, data)); }
function b64_hmac_md5(key, data) { return binl2b64(core_hmac_md5(key, data)); }
function str_hmac_md5(key, data) { return binl2str(core_hmac_md5(key, data)); }
/*
* Perform a simple self-test to see if the VM is working
*/
function md5_vm_test()
{
return hex_md5(“abc”) == “900150983cd24fb0d6963f7d28e17f72”;
}
/*
* Calculate the MD5 of an array of little-endian words, and a bit length
*/
function core_md5(x, len)
{
/* append padding */
x[len >> 5] |= 0x80 <>> 9) << 4) + 14] = len;
var a = 1732584193;
var b = -271733879;
var c = -1732584194;
var d = 271733878;
for(var i = 0; i 16) bkey = core_md5(bkey, key.length * chrsz);
var ipad = Array(16), opad = Array(16);
for(var i = 0; i > 16) + (y >> 16) + (lsw >> 16);
return (msw << 16) | (lsw & 0xFFFF);
}
/*
* Bitwise rotate a 32-bit number to the left.
*/
function bit_rol(num, cnt)
{
return (num <>> (32 – cnt));
}
/*
* Convert a string to an array of little-endian words
* If chrsz is ASCII, characters >255 have their hi-byte silently ignored.
*/
function str2binl(str)
{
var bin = Array();
var mask = (1 << chrsz) – 1;
for(var i = 0; i >5] |= (str.charCodeAt(i / chrsz) & mask) << (i%32);
return bin;
}
/*
* Convert an array of little-endian words to a string
*/
function binl2str(bin)
{
var str = "";
var mask = (1 << chrsz) – 1;
for(var i = 0; i >5] >>> (i % 32)) & mask);
return str;
}
/*
* Convert an array of little-endian words to a hex string.
*/
function binl2hex(binarray)
{
var hex_tab = hexcase ? “0123456789ABCDEF” : “0123456789abcdef”;
var str = “”;
for(var i = 0; i >2] >> ((i%4)*8+4)) & 0xF) +
hex_tab.charAt((binarray[i>>2] >> ((i%4)*8 )) & 0xF);
}
return str;
}
/*
* Convert an array of little-endian words to a base-64 string
*/
function binl2b64(binarray)
{
var tab = “ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/”;
var str = “”;
for(var i = 0; i > 2] >> 8 * ( i %4)) & 0xFF) <> 2] >> 8 * ((i+1)%4)) & 0xFF) <> 2] >> 8 * ((i+2)%4)) & 0xFF);
for(var j = 0; j binarray.length * 32) str += b64pad;
else str += tab.charAt((triplet >> 6*(3-j)) & 0x3F);
}
}
return str;
}
Sorry, somehow the copy paste action of the script went wrong….copied again below
/*
* A JavaScript implementation of the RSA Data Security, Inc. MD5 Message
* Digest Algorithm, as defined in RFC 1321.
* Version 2.1 Copyright (C) Paul Johnston 1999 – 2002.
* Other contributors: Greg Holt, Andrew Kepert, Ydnar, Lostinet
* Distributed under the BSD License
* See http://pajhome.org.uk/crypt/md5 for more info.
*/
/*
* Configurable variables. You may need to tweak these to be compatible with
* the server-side, but the defaults work in most cases.
*/
var hexcase = 0; /* hex output format. 0 – lowercase; 1 – uppercase */
var b64pad = “”; /* base-64 pad character. “=” for strict RFC compliance */
/* var chrsz = 8; */ /* bits per input character. 8 – ASCII; 16 – Unicode */
var chrsz = 16; /* bits per input character. 8 – ASCII; 16 – Unicode */
/*
* These are the functions you’ll usually want to call
* They take string arguments and return either hex or base-64 encoded strings
*/
function hex_md5(s){ return binl2hex(core_md5(str2binl(s), s.length * chrsz));}
function b64_md5(s){ return binl2b64(core_md5(str2binl(s), s.length * chrsz));}
function str_md5(s){ return binl2str(core_md5(str2binl(s), s.length * chrsz));}
function hex_hmac_md5(key, data) { return binl2hex(core_hmac_md5(key, data)); }
function b64_hmac_md5(key, data) { return binl2b64(core_hmac_md5(key, data)); }
function str_hmac_md5(key, data) { return binl2str(core_hmac_md5(key, data)); }
/*
* Perform a simple self-test to see if the VM is working
*/
function md5_vm_test()
{
return hex_md5(“abc”) == “900150983cd24fb0d6963f7d28e17f72”;
}
/*
* Calculate the MD5 of an array of little-endian words, and a bit length
*/
function core_md5(x, len)
{
/* append padding */
x[len >> 5] |= 0x80 <>> 9) << 4) + 14] = len;
var a = 1732584193;
var b = -271733879;
var c = -1732584194;
var d = 271733878;
for(var i = 0; i 16) bkey = core_md5(bkey, key.length * chrsz);
var ipad = Array(16), opad = Array(16);
for(var i = 0; i > 16) + (y >> 16) + (lsw >> 16);
return (msw << 16) | (lsw & 0xFFFF);
}
/*
* Bitwise rotate a 32-bit number to the left.
*/
function bit_rol(num, cnt)
{
return (num <>> (32 – cnt));
}
/*
* Convert a string to an array of little-endian words
* If chrsz is ASCII, characters >255 have their hi-byte silently ignored.
*/
function str2binl(str)
{
var bin = Array();
var mask = (1 << chrsz) – 1;
for(var i = 0; i >5] |= (str.charCodeAt(i / chrsz) & mask) << (i%32);
return bin;
}
/*
* Convert an array of little-endian words to a string
*/
function binl2str(bin)
{
var str = "";
var mask = (1 << chrsz) – 1;
for(var i = 0; i >5] >>> (i % 32)) & mask);
return str;
}
/*
* Convert an array of little-endian words to a hex string.
*/
function binl2hex(binarray)
{
var hex_tab = hexcase ? “0123456789ABCDEF” : “0123456789abcdef”;
var str = “”;
for(var i = 0; i >2] >> ((i%4)*8+4)) & 0xF) +
hex_tab.charAt((binarray[i>>2] >> ((i%4)*8 )) & 0xF);
}
return str;
}
/*
* Convert an array of little-endian words to a base-64 string
*/
function binl2b64(binarray)
{
var tab = “ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/”;
var str = “”;
for(var i = 0; i > 2] >> 8 * ( i %4)) & 0xFF) <> 2] >> 8 * ((i+1)%4)) & 0xFF) <> 2] >> 8 * ((i+2)%4)) & 0xFF);
for(var j = 0; j binarray.length * 32) str += b64pad;
else str += tab.charAt((triplet >> 6*(3-j)) & 0x3F);
}
}
return str;
}
Nope, somehow submitting a copy of the script does not work properly. Any suggestions?
Hi Phillip,
I’m playing around with Perl / FB7270 / Tracing and the new SID Login variant.
1. From an external PC I’m able to succesully login via Perl script or manualy via a Telnet:80 session. This works fine with the FB Password enabled and disabled.
2. Not working are the exact same http commands from (1) when entered directly on the FB7220 terminal / telnet window via perl / wget. Always a new seed is returned and an SID=”0000000000000000″
– The MD5 Hash is exactly identical as calculated on the PC.
– The FB returns “wrong user and Passwort” while trying to get the SID via the http-POST.
– The result is identical once the FB password is disabled.
=> Were you able to complete a succesful login directly on the FB?
Viele Grüße, Uwe
Hi Uwe,
To be honest, I’m not sure I follow. Are you using my script or are you using a script of your own. If you’re using your own script and your Fritz!Box is similar than mine, the script must:
– retrieve the ‘challenge’ from the login.lua page
– transform your password (strip umlauts and stuff)
– calculate the challenge response using the password
– post a response to login.lua with the ‘response’ parameter set
– and then parse the ‘sid’ parameter from the response HTTP ‘Location’ header
Does it do that?
Cheers
Philipp
Hi,
I have a 7360 and since I updated to the most recent firmware (111.05.51) the “Rebuild index” link on the router webpage is gone. AVM support could not fix the issue.
So your script looks double interesting to me.
One question: what is the site at URL sd2dav.1und1.de that you put into the sendRequest call?
Thanks
Well the script uses the frontend. So if AVM is not completely stupid, they removed the functionality in the backend and did not just remove the link.
What URL are you talking about?
I agree with you, the feature is probably gone.However, Avm support made a different statement saying:”Sorry, we can’t fix it, we suggest to wait for the next fw update”. I wonder if they talked to R&D.
Sorry, I misused the term URL as it is not the correct way to identify what I was talking about.
At line 60 of your script, the sendRequest call is:
“nas_activ=on&internal_mem_activ=on&viewUsb=on&usb_disconnect=0&webdav_url=https%3A%2F%2Fsd2dav.1und1.de&webdav_username
What is the server “sd2dav.1und1.de” in it?
Thanks
Honestly I don’t know what this part is, it does not even seem to be in the UI anywhere. I think there was (is?) a functionality to connect remote WebDAVs from public providers such as 1&1, GMX, etc. (German ones). I’ll send you the HTML source code of my settings.lua and a screenshot. Maybe that helps.
Thanks,
that would intersting
Bye
Sergio
Hello again,
apparently AVM auomated the mediaserver database creation in the recent firmware releases: if the database was not created yet, it will be built when you first attempt to access the mediaserver via a DLNA client.
The database is a sqlite DB contained in file fritznasdb_part.db3.
The database can be accessed using a sqlite3 client. The datamodel is fairly simple (13 tables)
I managed to update it to let new video files added to an existing folder on the mediaserver be visible from a DLNA client without rebuilding the index (it involved inserting records in two tables only: “video” and “upnp_video_container”).
I’d like to create a simple web page that automates via php the entire process (browse new added file, extract properties and update the fritz database).
With some effort it could be possible to write a routine that refreshes or rebuilds the entire index.
Hi Philipp,
http://blog.philippheckel.com/uploads/2012/12/fritzbox-md5-2.1-BROKEN.js.php returns a 0-byte file for me.
Is this intentional?
Cheers,
M
Nice post,
eventually found the md5-part can be built with the standard toolchain: https://home.debian-hell.org/blog/2013/05/13/update-konfiguration-der-avm-fritzbox-7390-per-wgetcurl-script-sichern/
Wow this looks a lot(!) easier!
I also fixed the broken MD5 link.
@Ronald Janseen
In order for the script to authenticate with a 7360 you need to make a slight modification to the code. In fact the challenge token in the lua page is assigned to a variable named g_challenge:
g_challenge = “”
To extract it you have to modify the regex handling at lines 39 and 40 and replace Philipps code with:
if (preg_match_all(‘/^g_challenge = “([^”]+)”/m’, $login_response, $m)) {
$challenge = $m[1][0];
However, as I mentioned in one of my previous comments, the refresh feature seems to have been removed from the latest 7360 firmware (the link to manually rebuild the index is not on the USB device list page anymore) and even the script executes successfully, it has no apparent effect on my device.
Hi guys,
I’m a user, not a programmer. So, I don’t understand the details of the great work you’re doing here. But:
I did some things in the past: e.g. I hacked my Samsung TV installing SamyGo, I modified and copied some scripts I needed for my NAS, i modified my old Motorola Cable-Modem to enable reconnect a.s.o.
Normally, you get access using putty (SSH or telnet) or flashing hardware using USB-sticks. I avoid using Linux, but if it has to be, I also use this.
I’ve got a FritzBox 6360 cable. I really like Philipps script, I’m very interested to install it. Reason is, that I’ve got a webcam sending pictures via ftp to the 6360 NAS (USB stick). I’m able to browse the pictures via DLNA from my TV, but because the 6360 refreshs the DLNA table just once a day, the pictures are not really “live”. (not talking about the modded Samy, there’s no problem at all)
There is one thing I really don’t understand:
I browsed the web for about 1 hour, but I wasn’t able to find out HOW DO YOU GET ACCESS TO YOUR *!@!!! 6360?!!?
If I’m not totally stupid, nearly any hacker, programmer, user etc. states that the 6360 is one of the few, perhaps the ONLY!! AVM FritzBox that can’t be accessed via telnet. Ok, you can try to change some things via USB, e.g. change firmware, perhaps even install Freetz. One guy explained after opening the box there seems to be a hidden, disabled serial access you have to build a connector for. Another guy tried to play around a little bit modifiing the saved settings file, another one with browsers developing kits getting access to 3 or 4 “hidden” options-sites. But more or less, most of these people have just been able to change options you’re also able to change via WebGUI.
@Philipp:
what does your script do? Is the idea that it’s getting acess to the Box using a second device (PC, Laptop, Tablet) and refreshs the DLNA index? If it’s like this i want to mention, that the “perfect” solution, the goal of most programmers is the implementation of the script into the device it is programmed for. For sure, I could copy the script to one of my NAS servers running 24/7, setting a cronjob to start it every 2 minutes, working fine. 4-8 HDDs waking up every 2 minutes to start a 5 kb script… :-(
If I read Sergios posts, I really get jealous: modify “file fritznasdb_part.db3″… Unlike Sergios 7360 there’s no way to get access to the root folder or the OS scripts of the 6360 and modify sth. “in the heart” of the box.
I really hope I missed sth., but I think, there is no easy way to configure the 6360 that way, that it actualises the DLNA table at every file change, or perhaps every 5 minutes or so. Besides Freetz, perhaps. But German boxes are always rent, a massive mod like installing another OS could result in law problems.
Please, PLEASE correct me if I’m wrong, I prefer being embarassed rather than fail.
Philipps script is nearly the one I need, but it has to be unstalled on the 6360, otherwise I could tell the cam directly to save the pictures on the NAS. That’s no option because of energy, costs, nature. Thx people for listening and your great work.
Nach dem Security Update der FritzBox 7330 lief das Script nicht mehr. Folgende Änderung hat’s wieder funktionstüchtig gemacht:
Ändere
if (preg_match(‘!\[“security:status/challenge”\] = “([^”]+)”,!’, $login_response, $m)) {
in
if (preg_match(‘/g_challenge = “([^”]+)”/’, $login_response, $m)) {
Sorry! In english:
After the Security Update of my FritzBox 7330 the Script didn’t work anymore. The following change fixed it for me:
Change
if (preg_match(‘!\[“security:status/challenge”\] = “([^”]+)”,!’, $login_response, $m)) {
to
if (preg_match(‘/g_challenge = “([^”]+)”/’, $login_response, $m)) {
Hi, is the script working for anyone with FRITZ!OS 06.30?
Also with the change from Stephan it is not working for me:
[GET login.lua ] Retrieving login page and challenge ... challenge is 8b97ad26, calculated response is 8b97ad26-65803813083b3bd867959dd6231e255c
[POST login.lua ] Logging in ... postlogin_response does not match.
Sorry :( Doesn’t work for me either anymore. Don’t know when it quit working, but I always get messages like this:
[GET login.lua ] Retrieving login page and challenge … challenge is 9f79975b, calculated response is 9f79975b-06a571a07a6d048b52fa0e3788d20641
[POST login.lua ] Logging in … postlogin_response does not match.
I do not understand this.
I’ve created a hex_md5 script containing
#!/usr/bin/php
which gives me for example:
$ ./hex_md5 abcdefghijklmnop
507fb100efe8930ac6378cc3078f5e9f
while, when I execute in FireBug, while on http://fritz.box/login.lua
hex_md5(“abcdefghijklmnop”)
I get
a7d7c122f716c6c0ffba377c421e34b9
So something strange is going on here :(
SOLVED!
Marcus’ hint above gave the correct clue.
1. Remove the broken library by commenting or removing the line
require(dirname(__FILE__).”/fritzbox-md5-2.1-BROKEN.js.php”);
2. Change the line
$response = $challenge.”-“.BROKEN_hex_md5($response);
to
$response = $challenge.”-“.md5(iconv(“UTF-8”, “UTF-16LE”,$response));
That solved it for me
Apparently AVM have fixed their broken MD5 implementation. I had ported it from JS to PHP to make it work. Glad it worked out for you.
To be honest: I do not think, they have fixed anything. When I compared the md5_core function (or maybe the bug isn’t in there?) to what you’ve ported, I don’t see any difference.
I have fritz box 6360 and 6320, we are going on provision system to plan activate this box on docsis 3.0, after plan change it should reboot automatically , but fritz is not doing as well , but other vendor is doing same . we need the mib for reboot the modem please help us ..