File access errors

For general web development questions that are not specifically related to CSS HTML Validator. This includes (but is not limited to) general HTML, CSS, Accessibility, JavaScript, and SEO questions.
Post Reply
User avatar
Lou
Rank V - Professional
Posts: 299
Joined: Fri Jul 29, 2005 5:55 pm
Location: CO
Contact:

File access errors

Post by Lou »

{Let me preface with PHP may not be the ideal language for this file access heavy application. - This started as a "simple" winter why-not project that has gotten out of hand.}

The objective of the current application is to compare a "new" file with many others to see if it is unique. Running the script hours on end, opens a file to test against thousands of times. Randomly, daily-every other day, the script stops with a error. Today I managed to recover.
/WDMYCLOUD/my/zobrist/Data/step3a/variant_826RHG.txt ADDED as //WDMYCLOUD/my/zobrist/towers/tower_k3fya.txt * * *

Warning: fopen(//WDMYCLOUD/my/zobrist/towers/tower_k3apw.txt): Failed to open stream: Invalid argument in C:\xampp\htdocs\zobrist\duplicate.php on line 30
duplicate #30a: //WDMYCLOUD/my/zobrist/towers/tower_k3apw.txt
//WDMYCLOUD/my/zobrist/Data/step3a/variant_826RHH.txt ADDED as //WDMYCLOUD/my/zobrist/towers/tower_k3fyb.txt * * *
The first and last lines above are normal output from elsewhere.
The related code is:
Code segment
Code segment
Tcode1.gif (15.25 KiB) Viewed 131 times
Code lines #24-28 check that $dir2 and $test_name point to a valid file. The <fopen> on line #30 generated the Warning: this time, BUT after a 5 second delay the similar line #35 worked fine and the script continued to line #44.

Within the error condition the routine towers_added() tries to save the current state to aid recovery.

I'm using Windows 11, FireFox, both current. Locally have XAMMP installed for PHP & mySQL, again current. The //WDMYCLOUD/ is a 6TB hard drive on my LAN. In the PHP.ini file I have increased the "default_socket_timeout" from 60 seconds to 120 with no effect.

Before adding the <sleep(5)> etc., with a Warning the script would go directly to the towers_added() routine, the first <fopen()> in the routine would also create a Warning and then crash when trying to writing to a file. Any ideas? Or am I just stressing Windows with thousands of file operations?

The TLTR part is: I have a Zobrist puzzle, (similar to Soma a 3x3x3 puzzle), that can be a 3x3x15 tower
One of many solutions
One of many solutions
ztower.jpg (23.62 KiB) Viewed 131 times
I wrote a script to find solutions stacking the 33 puzzle pieces into a 3x3x15 tower. After the script found 1,000 solutions I started checking for duplications and finding variants of solutions I had. Currently we have 96,000+ unique answers and am checking some 431K third-generation variants.

I have divided the 96K answers into 161 classes, but even if they were all the same size that would be almost 600 answers (files) to check to be sure a new solution was unique. There are many classes that have only a few members so each new answer (file) needs to be check against 1,000+ other files. Currently the most active classes have 2-5K members. As I said this "why-not" exercise has gotten out of hand!
Lou
Say what you will about Sisyphus. He always has work.
User avatar
Albert Wiersch
Site Admin
Posts: 3786
Joined: Sat Dec 11, 2004 9:23 am
Location: Near Dallas, TX
Contact:

Re: File access errors

Post by Albert Wiersch »

Windows can only take so much... :D

What I have done for issues like this is keep retrying up to a certain number of times... like put it in a while loop and if it can't open the file, wait a few milliseconds or seconds, increment a retry counter, and run the loop again... and keep trying until the retry counter hits its limit or the operation is successful (in which case break out of loop).

Also make sure every file is being closed properly and you're not leaving them open.

And could Windows be doing something in the background that limits its resources? Maybe there is a virus scanner that has to check every file open request and that is causing issues? Maybe something else opened that file for reading and/or writing at the same time?

Other than that, I can't think of anything.

I can also tell you I know nothing about Zobrist puzzles, but it looks interesting. :D
Albert Wiersch, CSS HTML Validator Developer • Download CSS HTML Validator FREE Trial
User avatar
Lou
Rank V - Professional
Posts: 299
Joined: Fri Jul 29, 2005 5:55 pm
Location: CO
Contact:

Re: File access errors

Post by Lou »

Thanks for the list of things to check. I hadn't thought about what Norton may be doing in the background and who knows what bloated Windows is doing!
Windows getting behind clearing some temp space was my first thought. The large network hard drive is several years old and a little sluggish by newer standards. However, Watching the number of files grow faster than expected, I decided the hard drive was a better option than the more limited space on the lap top SSD for the 10s of thousands of small files.

Thanks for the conformation that I'm on the right track. Time to clean up the patchy quick fix for something better.
Lou
Say what you will about Sisyphus. He always has work.
User avatar
Lou
Rank V - Professional
Posts: 299
Joined: Fri Jul 29, 2005 5:55 pm
Location: CO
Contact:

Re: File access errors

Post by Lou »

Just a followup. I replaced the script fragment above with the following as suggested (lines 29-38):
Tcode2.gif
Tcode2.gif (15.78 KiB) Viewed 84 times
The result seems to be that this along have changed the timing enough to avoid file open errors.
Note line # 33. the message has not shown up in the output during processing ~24/7 since last post.

Thanks. Feed the Trolls.
Lou
Say what you will about Sisyphus. He always has work.
User avatar
Albert Wiersch
Site Admin
Posts: 3786
Joined: Sat Dec 11, 2004 9:23 am
Location: Near Dallas, TX
Contact:

Re: File access errors

Post by Albert Wiersch »

I'm glad it seems to be working! :)

I would recommend

Code: Select all

$name === false
instead of

Code: Select all

$fname == false
Albert Wiersch, CSS HTML Validator Developer • Download CSS HTML Validator FREE Trial
User avatar
Lou
Rank V - Professional
Posts: 299
Joined: Fri Jul 29, 2005 5:55 pm
Location: CO
Contact:

Re: File access errors

Post by Lou »

Yes
Lou
Say what you will about Sisyphus. He always has work.
Post Reply