Deep Sky Stacker - Excessive Stacking Time

[Startrek]

After using DSS for 4 years I’ve never had a stack that took longer than 30mins or so to complete ( with up to 4 or 5 hours of data )
Today it’s taken 2 hours to stack 160 subs with another 140 to go
Seems unusually long and never experienced this ever before ?

Camera is the ZWOASI2600MC
Stacking details -
300 x 60 sec subs ( 50MB fits ea )
40 x Flats ( 50MB fits ea )
40 x Bias ( 50MB fits ea )
DSS version 4.2.5 64 bit
Kappa Sigma clipping
Use all available processors checked
Star detection found 50 stars
Plenty of storage space in main and temporary ( 400GB )
i7 processor 2.60GHz
16GB ram
64 bit OS running Windows 10 Pro
No virus protection disabled
No notifications disabled
No updates disabled
Totally autonomous Laptop

I registered and stacked the 300 x 60sec light frames on their own without calibration frames and it took 30 minutes ?
How could adding calibration frames quadruple the stacking time ?

Any ideas or suggestions or tips to speed up the process
??
Admittedly I’ve never ever stacked any more than 120 subs previously over the 4 years ( firstly with a DSLR and in the last 14 months with the 2600MC) mainly because subs have been 90 sec to 300 sec +
I chose 60 sec subs due to galaxy imaging in Bortle 8 skies with Neigbours flood lights lighting my backyard up like downtown Las Vegas

Thanks

[glend]

Try reducing the star detection threshold , to say 25, and run the lights again to compare with the previous time taken.

[AdamJL]

Yep, Star Detection can increase the total time by a lot. You don’t need much for cooled cameras. Sometimes I run it at 2%! Which can give about 60 stars depending on the object

[Startrek]

I always adjust my Star Detection slider to 2% to start off with , then go up from there to reduce Star count
It was at 2% ( lowest ) for the 50 star count

At the moment DSS has stacked 245 in 4.5 hours with another 55 to go

This is ridiculous I’ve never had this stacking time ( the worst was 30 mins )

I’ll try another stack later on or tomorrow with say 25 stars as Glen suggested

Cheers
Martin

[Startrek]

Just finished stacking
Took 6hrs 20 minutes

Has anyone else had over 6hrs to stack that amount of data inc calibration frames ??

Thanks
Martin

[AdamJL]

Have you got "Use all available processors" ticked?

And just to check, you weren't running anything else in the background? You've restarted your PC recently as well (clear out RAM)?

[Pepper]

DSS took me ages last night to stack 40 odd subs from my dslr.
Autostakkert won’t stack moon shots from my asi533 without complaining of memory shortages.
I’ve just bit the bullet and decided to upgrade my computer.
Was i7 16gb but going to move up to i9 32gb and a new motherboard.

[Startrek]

Adam,
Read my original post regarding “ Use all available processors”
My lappy is standalone , purely Astro , I checked TaskManager to see what was running , only DSS and support programs
Last night I reduced the star count as advised by Glen (from 50 stars to 25 stars ) and stacking took 1 hr 45 mins instead of 6 hrs 20 mins
Then used a different set of flats with short exposures , reduced the stacking time again to only 45 mins
A quick stretch of each stacked image in Startools and all looked the same , field wasn’t exactly flat but usable
Things are looking a lot better
I think my lappy would benefit from more Ram too , 16GB is not huge , 32GB would be much better
Cheers
Martin

[Startrek]

Glen
Reducing star alignment count from 50 to 25 did the trick
Thanks
My lappy would run quicker with more Ram too , 16GB just isn’t enough today

Cheers
Martin

[glend]

Wonderful news.:thumbsup:

[AdamJL]

1hr 45 mins is still pretty slow though. What are the specs of your machine? (CPU i7 model specifically, DDR speeds etc).
Are you running off spinning rust (platter HDD) or a modern NVME SSD (or even a SATA SSD)

[Stephane]

Martin, with my extremely limited knowledge, I have noticed that DSS doesn’t like certain certain folder locations. Sometimes I take untracked milky way subs and stack a few hundred. If I take my images directly from the sdd card rather than a folder on my desktop, time increases dramatically for some reason (like from 30min to a few hours). The same happened with Sequator. Not sure if it’s just me or has anything to do with my ancient sdd card, but now I always place my folders on the desktop.

Cheers,
Stéphane

[Nikolas]

The 2600 images are huge as well and you stacked quite a lot

[Startrek]

Stephane
Thanks for your experiences
All my captured data is mapped directly to my laptop, even in my DSLR days using BYEOS it all went straight to specific folders
I think the time lag its more to do with this unbelievable camera I use ( 2600MC ) subs are huge fits files ( 50MB each ) and I stacked nearly 400 including calibration files
I seem to have found a solution which worked well today , reduce star detection by half and the whole job stacked in 45mins which is normal
Just processing the image this afternoon (NGC 253 Sculptor Galaxy) so you will be able to see when I post the image later on what’s capable from only a 6” f6 Bintel newt and these latest unbelievable ZWO OSC cameras ( bearing in mind I’m imaging under Bortle 7/8 Sydney skies with 2 storey houses either side of me lit up like Las Vegas ) not exactly ideal conditions for Astrophotography
Thanks
Cheers
Martin

[Startrek]

Nikolas,
Yep very true
Huge amounts of data
It’s stacking much better today as I took Glens advice and reduced the Star detection by half ( now only 25 stars ) DSS only needs 8 to 10 stars to align and stack
Thanks
Martin

[Stephane]

Thanks Martin, I am quite excited to see the result. I am still strongly hesitating between the Bintel f/5 and f/6 despite extensive research. I hear coma disappears almost completely with an f/6 meaning I won't need a coma corrector.

The only thing that concerns me is that the Bintel website bags its own f/6 OTA stating "It is still suitable for lunar and planetary imaging but not for imaging using larger sensors". ("Larger sensors"??). The slightly slower speed really doesn't bother me, as long as the optics are good. Looking forward to seeing your latest image.

Cheers,
Stéphane

[Startrek]

I still get a bit of coma with my f6 in the corners

Yes I read that info on the f6 Bintel newt 4 years ago ( ignored it ) I’ve even won a Bintel image of the month in Oct 2019 with my 6”f6 Bintel newt

The only modification I made to mine when I bought it 4 years ago was I added a dual speed focuser ( a must for AP ) don’t know if it comes with dual doors now ? And the second mod was fitting new heavy duty locally made Primary mirror springs after a year to make Collimation easier and hold Collimation longer

Cheers
Martin

[Stephane]

Congrats on the Bintel image of the month. That’s awesome. I am thinking of getting a ZWO EAF and just ask my capture software to refocus every increase of 5-10% of HFR. Not sure if I’m planning on going too heavy with automation from the start, but I thought that if I’ll get it down the track, I might as well get it early. Haven’t decided yet… Either that, or a peg lol. I might start with the peg…

[The_bluester]

Martin, one thing you might be able to change that would speed up individual processes to some degree. I would assume that the bias frames are the same set of 40 used over and over? I found the bias in my pair of 2600's too be very repeatable so I created a single master bias each for 0 gain and 100 gain. Same with flats, unless I disassemble the gear I create a master flat (Per filter for the 2600MC) and load the master instead of throwing all of the raw cal files at it every time. I have master darks too though I am experimenting on using the master bias instead.

I don't know how long it would add, but if you load the same 40 bias frames every time and it takes 10 minutes to process them, that is ten minutes every time you stack lights. Even if you are not saving a master bias or master flat when you process, DSS or whatever other software (Like APP which I use) is essentially creating a master bias/whatever else each time from the raw files you are loading and applying that, it is just not saving it to reuse later.

[Startrek]

Paul,
Thanks
DSS saves your cal frames into Masters for reuse which would save a bit of time
Unfortunately my rig in Sydney is not permanent ( I leave it out covered for a few weeks at a time ) plus it’s rare to get consecutive nights of imaging , so my camera is always removed and brought inside, so darks and bias would be ok
Down south at my Obs I could utilise Masters more frequently
We spend about 7 months of the year in Sydney and 5 months down the south coast at our retirement / holiday home , but at random times due to ( grandkids , wife still casual teaching and so on .... even though I’m retired , I’m bloody busier than ever )
Cheers
Martin

[etill]

You may be on the right track with the ram, once the systems has to start swapping it out to disk things get orders of magnitude slower. One of the things I like about pixinsight is that it keeps track of system resources it has available and schedules its work to avoid hitting bottlenecks. I don't have DSS to compare, so can't comment on that one.

Are you able to check the memory usage during stacking in the task manager? I mainly use mac and linux but windows refers to swap space as the page file, if there is any used during stacking it will cause a big bottleneck.

According to Google in performance monitor on windows you can track it (add counter -> page file -> usage), if none is being used then more ram may not make a difference. Another potential improvement is swapping to pci-e attached drives (many m.2 and all 2.5in SSD drives aren't). Using nvme, or pic-e (there's heaps of options now) will provide a significant I/O improvement even over SSDs with slower interfaces.

[Startrek]

Elliot,
Thanks for the comment
Appreciate folk following up ones problems
It’s all ok now , stacking is working well
A couple of things to take away from my issue -

1/ Stacking nearly 400 x 50MB files is a heavy load no matter what stacking software we use , I didn’t even consider that.

2/ My computer was coping well with the work load , here’s my memory status in task manager whilst stacking

Deep Sky Stacker
CPU 0.1 %
Memory at 325MB
Disk at 1.4 MB / sec

Total in Task Manager
CPU 15%
Memory at 28%
Disk at 9% to 14%

3/ Deep Sky Stacker was using all available processors (12)

4/ Reducing the Star detection by half ( 50 to 25 ) reduced the time from 6 hrs to 45 mins stacking time

The only drawback stacking huge quantities of subs from the 2600MC is that a 60 sec sub has the same file size as a 300 sec sub ( 50MB fits ) unlike Raw CR files which are tiny in comparison

Thanks

Martin