Newznab - config help etc. Not an invites thread. :)

Just got nn+ up and running in a centos6 vm 2cpus and 2 gb ram backfilling 120 days on 8 groups after initial download.
Currently running at around 300 Kbps and my raid 5 array on that vm has sat at around 6-8mbps for the last 48 hours.

Wishing I gave it 8gb of ram to start of with then backed off once back filled oh well will just have to wait for the backfill to finish.
 
You can stop the backfill and just restart it at any time. I sometimes stop it and then do an update_releases to see some of the updates without having to wait for the full backfill to complete.

Backfilling is not so heavy on memory, it is kore internet connection bound, from what I have seen but, the update_releases will benefit from more and a decent processor.

Setting a reasonible minimum size for releases under the group config also helps weed out a lot of trash.

RB
 
Upped ram to 4GB and gave it 4 cores (host is pretty underused), moved it onto a different raid set/lun.
Completed backfilling 120 days on 12 groups now going to 400 days.

Its really chewing on it now :) installed iotop to keep an eye on swap file usage and the vm seems a bit happier now.
 
Just in the process of setting this up now on a VM Ubuntu server, at the stage of entering some details in the web browser after the initial setup.
 
Cool, out of curiosity, how much space did you allocate to your VM running newznab and how much retention on the indexing are you using?
 
Last edited:
OK getting a bit further now, just got to the stage of adding a couple of groups, then running the update_binary.php and update_releases.php script but it didnt seem to pull anything

Code: 
Starting release update process (2013-01-24 22:00:33)
using latest regex revision 1742
Applying regex 2 for group alt.binaries.*
Applying regex 1 for group alt.binaries.*
Stage 2
Stage 3
Found 0 nfos in 0 releases
0 nfo files processed
Site config (site.checkpasswordedrar) prevented checking releases are passworded
Processing tv for 0 releases
Lookup tv rage from the web - Yes
Tidying away binaries which cant be grouped after 2 days
Deleting parts which are older than 2 days
Deleting binaries which are older than 2 days
Processed 0 releases
 
Last edited:
DJMK4 said:
Cool, out of curiosity, how much space did you allocate to your VM running newznab and how much retention on the indexing are you using?
Click to expand...

127gb, indexing 10 huge groups back to 50 days, using just over 7gb in total.

I'll probably go back another 50 to see if it scales with a plan to go to at least 365 days.
 
Ah cool thats not too bad.

Can anyone help on my post above? seem to be stuck when I run the update_releases script, doesnt seem to be pulling anything down, and nothing is then showing under my groups/releases etc, not sure why.

I have tried a few groups now, it updates the binaries ok, but just not the releases.
 
DJMK4 said:
Ah cool thats not too bad.

Can anyone help on my post above? seem to be stuck when I run the update_releases script, doesnt seem to be pulling anything down, and nothing is then showing under my groups/releases etc, not sure why.

I have tried a few groups now, it updates the binaries ok, but just not the releases.
Click to expand...

I noticed much the same on the free version of newznab. Apparently (so a friend tells me) it's because all the regex is out of date. (You can apparently find some more up to date regex out there on t'internet with a search).

The plus version doesn't have that problem as the devs have kept it much more up to date.

Whether that's your problem or not is a different matter :)
 
HarryPainful said:
Are you sure its pulling anything down during update_binaries.php?

The update_releases.php will do nothing if you have no binaries for it to run against.
Click to expand...

Yep you can see the binaries coming down as it takes a while, gives you a summary and look At the network graphs for that VM you can see a spike up when job starts when completes drops again

Infidelus said:
I noticed much the same on the free version of newznab. Apparently (so a friend tells me) it's because all the regex is out of date. (You can apparently find some more up to date regex out there on t'internet with a search).

The plus version doesn't have that problem as the devs have kept it much more up to date.

Whether that's your problem or not is a different matter :)
Click to expand...

Ta will have a search for this :)




EDIT: Had ago at updating the redex yesterday with the commands to pull of the SQL updates, and it worked by the looks, run the release script now and it pulls it down.
 
Last edited:
Looks like my db is screwed so going to remove all groups, delete and purge everything.

Can someone tell me what they have in global site settings for the following please.?
Header retention ?
Release retention ?
Max messages fetch at a time ?
Where to start days = ? days or ? posts

Once ive reset it all i'll pull down one group and let it index and add a group a day or until i see each one successfully display releases.
 
Going to start fine tweaking this today, and possibly looking at tying things up, taking a few backups and writing some quicknotes just incase everything goes pearshaped from here on

Will try and extend the backfill days as I go, dont want to screw it all up at this point and have to start again.
 
DJMK4 said:
Going to start fine tweaking this today, and possibly looking at tying things up, taking a few backups and writing some quicknotes just incase everything goes pearshaped from here on

Will try and extend the backfill days as I go, dont want to screw it all up at this point and have to start again.
Click to expand...

How are you getting on.? safebackfill script is working a treat for me and im no longer maxing our 4GB ram.
Just added some of the larger groups back into the db so total of 12 groups now and backfilled to 52 days, really makes a difference backfilling a day at a time and you can see the releases quicker too.

Much happier :) Just installing OpenVPN into my centos vm then i can access it from work along with everything else :)
 
You must log in or register to reply here.
Back
Top Bottom