cleverscope app memory leak

Hi,

just wondering if anyone else has been having this problem: with recent versions of the cleverscope software ( I don't recall having the problem in the past), I've been saving large sets of data to disk, a 400 frame sequence is pretty common. I find that each time I do this, the amount of memory consumed by the cleverscope increases, until the memory used is >1GB and the application crashes. It also won't quit after just saving one set.

I'm using the latest cleverscope version, 4.635, and it has been happening at least since 4.634.


One other minor problem- the number of frames no longer seems to save correctly in the binary format output by the application. It always seems set to 1.

Morgan

Hi, just wondering if anyone else has been having this problem: with recent versions of the cleverscope software ( I don't recall having the problem in the past), I've been saving large sets of data to disk, a 400 frame sequence is pretty common. I find that each time I do this, the amount of memory consumed by the cleverscope increases, until the memory used is >1GB and the application crashes. It also won't quit after just saving one set. I'm using the latest cleverscope version, 4.635, and it has been happening at least since 4.634. One other minor problem- the number of frames no longer seems to save correctly in the binary format output by the application. It always seems set to 1. Morgan

Hello morgan, we will check this out. We have been playing quite a bit with storage and memory in the last couple of versions (and in the next, due out shortly) so maybe we have inadvertantly done something. With the number of frames, are you sure you are not looking at the frames per capture value? If the number of frames is set to 2 (the default), even if you set the frames per capture to something larger (eg 10), it will set back to 1 when you load the file again (because it always has to be at least 1 less than the number of frames). Our playing seems to have the number of frames working correctly.

Have you been saving with text or binary? The next version improves the text saving and loading a great deal. We will post when it is on the web site.

Hello morgan, we will check this out. We have been playing quite a bit with storage and memory in the last couple of versions (and in the next, due out shortly) so maybe we have inadvertantly done something. With the number of frames, are you sure you are not looking at the frames per capture value? If the number of frames is set to 2 (the default), even if you set the frames per capture to something larger (eg 10), it will set back to 1 when you load the file again (because it always has to be at least 1 less than the number of frames). Our playing seems to have the number of frames working correctly. Have you been saving with text or binary? The next version improves the text saving and loading a great deal. We will post when it is on the web site.

Hi Bart,

thanks for the quick reply. The files I'm saving are your .bin format as I need to manipulate them externally, and saving as text makes them far too large.

The program does exactly what I want, and saves the 400 frames per capture (or whatever number I set it to. Total frames is set to 401 in this case). However, in the binary header, ""Fn"" is set to 1, and ""n"" is set to a big number, which happens to be 400 x the number of points in each frame. So I'm currently just overriding this value with the number of frames that I know I saved, and everything works fine. It's just a little bit annoying that I have to separately keep track of how many frames I saved, that's all.

I hope it's an easy fix!
Morgan

Hi Bart, thanks for the quick reply. The files I'm saving are your .bin format as I need to manipulate them externally, and saving as text makes them far too large. The program does exactly what I want, and saves the 400 frames per capture (or whatever number I set it to. Total frames is set to 401 in this case). However, in the binary header, ""Fn"" is set to 1, and ""n"" is set to a big number, which happens to be 400 x the number of points in each frame. So I'm currently just overriding this value with the number of frames that I know I saved, and everything works fine. It's just a little bit annoying that I have to separately keep track of how many frames I saved, that's all. I hope it's an easy fix! Morgan

Aah - I do know about that one, and another person wrote us an email about it. We are getting ready to release 6436, and it has been fixed in that. Before we do, we'll have a very good look for memory leaks. Eventually (maybe 6437) we'll make it so you can save the whole frame array in one go. The ability to upload it, and store it has been there for ages, while we figure how to make the user interface friendly enough to make it useful (our end goal is colour coded time variant spectral displays, and time variant envelope displays, and packetized protocol decoders. We'll get there eventually).

Aah - I do know about that one, and another person wrote us an email about it. We are getting ready to release 6436, and it has been fixed in that. Before we do, we'll have a very good look for memory leaks. Eventually (maybe 6437) we'll make it so you can save the whole frame array in one go. The ability to upload it, and store it has been there for ages, while we figure how to make the user interface friendly enough to make it useful (our end goal is colour coded time variant spectral displays, and time variant envelope displays, and packetized protocol decoders. We'll get there eventually).

Great, thank you!

I'll inevitably be doing more 'testing' on the memory issue myself in the next few days. I'll post if I notice anything else.

Morgan

Great, thank you! I'll inevitably be doing more 'testing' on the memory issue myself in the next few days. I'll post if I notice anything else. Morgan

Just an update-

I tried to repeat the memory leak thing artificially, as I'm not running my experiment this week, and was actually finding it not so easy to do. Just downloading a lot of frames doesn't seem to do it. Having the maths and spectrum graphs open as I usually do, as well as downloading about 400 shots seems to use up a lot of memory which doesn't seem to be freed, but it's still far short of the runaway behaviour I was talking about.

Not sure!

Morgan

Just an update- I tried to repeat the memory leak thing artificially, as I'm not running my experiment this week, and was actually finding it not so easy to do. Just downloading a lot of frames doesn't seem to do it. Having the maths and spectrum graphs open as I usually do, as well as downloading about 400 shots seems to use up a lot of memory which doesn't seem to be freed, but it's still far short of the runaway behaviour I was talking about. Not sure! Morgan

Hello Morgan,
Ok we have an update on the website, which tidies up a number of areas, and adds some new trigger functionality. As part of that we tested the binary save (and the text save and load on large data sets). We could not find any memory leaks. As you say, once space is allocated in the array we leave it there, rather than de-allocate it. We tossed around on this, and came to the conclusion that if a user has used a large data set, they are likely to do it again, and so leaving the data space allocated, and re-using it is much more efficient, and faster than releasing it, and reallocating it. We don't allocate the space in the first place until the user does something that is large.

Our steps:
We setup a cleverscope to capture 2900 frames, 2901 frames were allocated. We used the sig gen as the source, with auto stepping in the acquisition unit (so we could see the things were as they should be). It was an 8M CS328A. The scope graph width was 1.5 ms, 1390 samples. The Sig Gen frequency step was 100 Hz. The start frequency was 1kHz. Final frequency was 291 kHz. Here are the memory values we found:
Application opened, no sampling - 133.8M
Opened acquisition unit, captured 2900 frames, and transferred one frame back - 139.5M
Set the Frame transfer drop down to 'Seq' (next to the get frame button).
Did a Get Frame. Samples in PC memory = 4031000. PC memory = 203.8M
Did a Save Graph as Binary to save the entire frame set to disk. PC memory = 254.3M
The size of each channels binary file was 15,626 kB
Repeated this process 10x.
After 10x saves, PC memory was - 254.4M

We didn't see any memory leaks. The binary save now includes the number of samples in one frame, the number of frames, and the current (ie last) frame number.

I hope this helps.

- Bart

Hello Morgan, Ok we have an update on the website, which tidies up a number of areas, and adds some new trigger functionality. As part of that we tested the binary save (and the text save and load on large data sets). We could not find any memory leaks. As you say, once space is allocated in the array we leave it there, rather than de-allocate it. We tossed around on this, and came to the conclusion that if a user has used a large data set, they are likely to do it again, and so leaving the data space allocated, and re-using it is much more efficient, and faster than releasing it, and reallocating it. We don't allocate the space in the first place until the user does something that is large. Our steps: We setup a cleverscope to capture 2900 frames, 2901 frames were allocated. We used the sig gen as the source, with auto stepping in the acquisition unit (so we could see the things were as they should be). It was an 8M CS328A. The scope graph width was 1.5 ms, 1390 samples. The Sig Gen frequency step was 100 Hz. The start frequency was 1kHz. Final frequency was 291 kHz. Here are the memory values we found: Application opened, no sampling - 133.8M Opened acquisition unit, captured 2900 frames, and transferred one frame back - 139.5M Set the Frame transfer drop down to 'Seq' (next to the get frame button). Did a Get Frame. Samples in PC memory = 4031000. PC memory = 203.8M Did a Save Graph as Binary to save the entire frame set to disk. PC memory = 254.3M The size of each channels binary file was 15,626 kB Repeated this process 10x. After 10x saves, PC memory was - 254.4M We didn't see any memory leaks. The binary save now includes the number of samples in one frame, the number of frames, and the current (ie last) frame number. I hope this helps. - Bart