To post a new support question, click the Post New Topic button below.
Current Version: 4.8.5 | Sytist Manual | Common  Issues | Feature Requests

Please log in or Create an account to post or reply to topics.
You will still receive notifications of replies to topics you are part of even if you do not subscribe to new topic emails.

Upload Process - Fatal Memory Error

H
40 posts
Tue Dec 01, 20 12:22 AM CST
Trying to understand why this message started happening AFTER already processing over 4000 images today on a working Sytist system 3.7.2. This system has been successfully processing large files, and large quantities of large files, for quite some time (since Tim walked me through config files a year or more ago). But now it seems to be at a standstill for importing new files that I've FTP'd into directories for a gallery.

Message:
------
Fatal error: Out of memory (allocated 634650624) (tried to allocate 21856 bytes) in process-ftp-upload.php on line 197
-------
Realize it's a memory error from php posting process....but been here before with larger files, set all this up. It's been working on large files......not sure what changed.

This is a self hosted system, I own the server, and control it via WHM/cPanel (and SSH rarely). Short version....restarted all the services. Still get the error. Soft Reboot Server....Still get the error. Double checked all the config files, made some changes (see below). Still get the error.

Previously, the max file sizes were 10M. The memory_limit was set to -1 (which allows unlimited memory for php processes). But....since it quit working, tried the values below.

php.ini
-------
memory_limit -> 2048M
upload_max_filesize -> 30M
post_max_size -> 30M

sy-config.php
-------------
ini_set('memory_limit', '2048M');
ini_set('upload_max_filesize', '30M');
ini_set('post_max_size', '30M');

Server has 8gig memory, over 350 gig hard drive space free.

Any ideas? Or other info needed?
-blair
16,145 posts (admin)
Tue Dec 01, 20 6:26 AM CST
Sounds to me it is stuck on a really large photo that is just too large to process. 634650624 bytes is 605MB. I would check the next file to be processed.
Tim Grissett, DIA - PicturesPro.com
My Email Address: info@picturespro.com
H
40 posts
Tue Dec 01, 20 7:33 AM CST
Understood. I'm trying to understand the cause, so I can resolve the issue.

It was chugging through the Level 1 - Set 1 directory. Many files the same size. 10-11 megabytes each from a Canon R5. Screenshot below.

The folder that was remaining, and Sytist is currently working through, with zero problems, has files 10-16 megabytes each. No memory errors. Also have screenshot below.

Some of the files it has processed in this session, are over 20 megabytes.



Edited Tue Dec 01, 20 7:33 AM by Hart Photo
-blair
H
40 posts
Tue Dec 01, 20 7:38 AM CST
Screenshot of folder it failed.
Attached Photos

Edited Tue Dec 01, 20 7:40 AM by Hart Photo
-blair
This reply was deleted.
H
40 posts
Tue Dec 01, 20 7:39 AM CST
Screenshot of folder started working after the failure.
Attached Photos

Edited Tue Dec 01, 20 7:40 AM by Hart Photo
-blair
16,145 posts (admin)
Tue Dec 01, 20 7:55 AM CST
process-ftp-upload.php on line 197 is when it is trying to rotate a photo. So perhaps something about the image rotation takes a lot more CPU.
Tim Grissett, DIA - PicturesPro.com
My Email Address: info@picturespro.com
H
40 posts
Tue Dec 01, 20 8:08 AM CST
More on the story.....

I was able to rename the folder with 65 files it was failing on. Re-uploaded to sy-upload. Started the Upload process. Same memory error.

However......while Sytist/PHP is giving the memory error, the process IS importing the files into the system. Had to click on the browser refresh on each error (or just wait for it to refresh on it's own). 65 refreshes, 65 files uploaded and showing in the admin.
-blair
H
40 posts
Tue Dec 01, 20 8:28 AM CST
------
process-ftp-upload.php on line 197 is when it is trying to rotate a photo. So perhaps something about the image rotation takes a lot more CPU.
------

Ok....so these are files that were originally captured vertically (and don't need to be rotated, unless there's something I'm missing). But...the system already processed a couple hundred other images successfully that were also captured vertically.



-blair
j
43 posts
Wed Feb 10, 21 9:54 AM CST
Blair, Did you ever find a fix for this? I'm kind of having a similar issue. All of a sudden can't process 10-12mb files.

[09-Feb-2021 17:09:56 America/New_York] PHP Fatal error: Out of memory (allocated 134217728) (tried to allocate 28672 bytes) in /services/webpages/m/c/mcdphoto.net/secure/sy-admin/photo.process.functions.php on line 136

Host has a large limit on uploads, and was able to upload large files via FTP, but they don't get processed.

Thanks for an info you have. J
H
40 posts
Wed Feb 10, 21 1:49 PM CST
@Jeremy.....

My problem wasn't necessarily resolved, just worked through I guess. I was only having the memory issue on large files captured vertically. I haven't done any large images vertically with the Canon R5 since originally posting the error message. My post repeated below hopefully helps:

-----
More on the story.....
I was able to rename the folder with 65 files it was failing on. Re-uploaded to sy-upload. Started the Upload process. Same memory error.
However......while Sytist/PHP is giving the memory error, the process IS importing the files into the system. Had to click on the browser refresh on each error (or just wait for it to refresh on it's own). 65 refreshes, 65 files uploaded and showing in the admin.
-----

So, I was able to import the large files ~12-18 mb each that were captured vertically. I get the error, but they were imported.

I am my own web hosting company....(my dedicated server is co-located in an ISP web hosting/data center). I can control all the PHP memory values Tim has mentioned to tweak. But....it seems some, not all of the large files generated by the Canon R5 captured vertically were causing my issue.

Sounds like your issue may be all large files now? If so, then it's possibly a change done for PHP values by your hosting company (or an automatic PHP upgrade done) that caused the new issue?

Hope this helps!
-blair
j
43 posts
Wed Feb 10, 21 2:37 PM CST
Blair, Thanks for your response to this issue. Super frustrating and slowing workflow WAY down.

We did upgrade to PHP 7.3 so I can use the texting feature in sytist. And this is when this all started. I have talked with my host a bunch and he says the limits are set way high. I'm not a server guru, so I guess the mem allocation is the big factor in this. I can and did set my ini.php files with the numbers below thinking that making them really large would help. Maybe still too small?

ini_set('memory_limit', '10000M');
ini_set('upload_max_filesize', '5000M');
ini_set('post_max_size', '5000M');

When I upload with the sysist uploader, they fail with a failed 500 error. If I upload them via FTP and try to process them, the browser just sit on "Processing Photos. Do not leave this window"
I can't think of any program settings I am missing.

The error log always comes up with this....
[10-Feb-2021 15:33:17 America/New_York] PHP Fatal error: Out of memory (allocated 134217728) (tried to allocate 28672 bytes) in /services/webpages/m/c/mcdphoto.net/secure/sy-admin/photo.process.functions.php on line 136

Thanks for your server expertise. If you have any tidbits for my host, then happy to relay this link to him. Thanks, J
12 total messages
Please log in or Create an account to post or reply to topics.
This post has been viewed 432 times
Category: Error
 
Loading more pages
Loading more pages

Sign up for email promotions.

Your information is safe with us and won't be shared.

Thank you for signing up!

 
©2003 - 2021 Grissett, LLC. All Rights Reserved.

By continuing to browse or by clicking Accept Cookies, you agree to the storing of cookies on your device necessary to provide you with the services available through our website.

    Accept   Privacy & Cookie Policy
Loading More Photos
Scroll To Top
Close Window
Loading
Close