-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Out of index issue and 'os' has no attribute 'EX_SOFTWARE' #33
Comments
Actually I executed above code in Jupyter notebook (in windows).The issue 'os' has no attribute 'EX_SOFTWARE' can be solved by replacing the os.EX_SOFTWARE with either 0 or 70. Can you please help me with out of index issue.Thanks in advance |
Hi Princec, Sorry for the delayed response. This error may be related to #29. I will leave this issue up for now and do some further testing to see if this is the case. Hopefully, a fix should be out by this week. I will make sure to update the issues. |
Thanks for quick reply. Actually when I am running the above code in google colab its working fine.But while running in the my local machine (windows with GPU) in Jupyter notebook i am getting error. Please let me know in case of any action or any changes required from my side. I have one more question while using it for image classification model
Please let me know if this format is correct ? Hoping to hear from you soon. |
Hello Princec! Yes - that is exactly the correct format :) |
Hello Kanding, Thank you so much for help. But one question here for below case if i want to see output after epoch 4 then i am getting output like this tracker = CarbonTracker(epochs=10, epochs_before_pred=4, monitor_epochs=4, 1875/1875 [==============================] - 3s 2ms/step - loss: 1.7441 - accuracy: 0.7232 CarbonTracker: But i feel output should be like below format Epoch 1/3 Followed by other outputs. Correct me if my understanding is wrong. |
Hi again, What framework are you using for your image classification model? This may be an error with your for-loop and the way from carbontracker import CarbonTracker
tracker = CarbonTracker(epochs=1, epochs_before_pred=0)
tracker.epoch_start()
model.fit(train_images, train_labels, epochs=3)
tracker.epoch_end() The output should be something like:
This treats your entire training session as a single "epoch" or iteration, while actually measuring the entire consumption (in this case the 3 epochs). Hope this helps. |
Hello, I am using tensorflow framework. It worked for me with below code
But when i am trying to change the value of epochs_before_pred and monitor_epochs to some other values (for example 2) then output is not generated. For Below case its working. Need something similar like below output.
Please let me know in case of any clarifications needed from my side. Hoping to hear from you soon |
Hi, Since you are using You can instead write your own training loop from scratch (see tensorflow.org/guide/keras/writing_a_training_loop_from_scratch) and then use the basic setup (see README.md) as you tried previously. |
Hello, Thank you so much for help. I will try that. Can we try to write logic for model.fit(epochs=EPOCHS) since most of times we use this only.(One of Enhancement). Please also check out of index issue in jupyter notebook(in windows machine with GPU) since i am able run same code in Google colab but getting out of index issue in local. Thanks and Regards, |
It may not be possible to add compatibility with non-loop structures like Thanks for your feedback. We appreciate it. :) |
Sure Thanks i will create separate issue with label feature request/enhancement. Thanks |
I have tried with new version of code. Still i am facing same issue
Please help me for same. |
Sorry about that. It looks like my last fix didn't quite do the trick. The latest commit 02a4a73 should fix this issue. You can test it by installing the package through the git repository:
|
Hello Again, Its running but giving zero value for everything I tried below test code mentioned by you.
Output of code
Please let me know if you need more information from my side. |
Could you try again and supply the argument You can replace the We really appreciate the feedback! |
Hi again Princec, There should be two log files. Could you also send the other one? |
Hello The updated output is
|
I am not sure what is causing this behavior. What operating system and Python version are you using? Could you also try running these code snippets and show me the output and log files?
and
|
I have Python 3.7.4 version and Windows 10.
|
It looks like NVML (we rely on pynvml) does not support the retrieval of power measurements on your specific GPU model. Unfortunately, this means that carbontracker cannot be used on your system. I will create another issue to add a descriptive error message when this is the case. I apologize that we did not see this sooner. |
Thanks for information. |
At the moment there are no workarounds to retrieve power measurements from your GPU, so unfortunately it is not possible. This is also unlikely to change unless NVIDIA introduces support for your specific GPU model in NVML. |
Congratulations for great work.
Issue 1 : Actually I tried below test code in jupyter notebook and getting below error.
Issue 2 --> Also faced same issue/ Error while running deep learning image classification model.
Hoping to hear from you soon.Thanks in advance
Error :
Trial: 0
CarbonTracker: The following components were found: GPU with device(s) GeForce GTX 1050 Ti.
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py", line 274, in epoch_end
self.tracker.epoch_end()
File "C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py", line 139, in epoch_end
self._log_epoch_measurements()
File "C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py", line 163, in _log_epoch_measurements
power_avg = np.mean(comp.power_usages[-2], axis=0)
IndexError: list index out of range
IndexError Traceback (most recent call last)
C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py in epoch_end(self)
273 try:
--> 274 self.tracker.epoch_end()
275
C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py in epoch_end(self)
138 self.epoch_times.append(time.time() - self.cur_epoch_time)
--> 139 self._log_epoch_measurements()
140
C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py in _log_epoch_measurements(self)
162 if np.isnan(power_avg).all():
--> 163 power_avg = np.mean(comp.power_usages[-2], axis=0)
164 self.logger.info(
IndexError: list index out of range
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
in
10 print("Trial: " + str(trial))
11 time.sleep(5)
---> 12 tracker.epoch_end()
13
14 tracker.stop()
C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py in epoch_end(self)
285 self._delete()
286 except Exception as e:
--> 287 self._handle_error(e)
288
289 def stop(self):
C:\ProgramData\Anaconda3\lib\site-packages\carbontracker\tracker.py in _handle_error(self, error)
326 self._delete()
327 else:
--> 328 sys.exit(os.EX_SOFTWARE)
329
330 def _output_energy(self, description, time, energy, co2eq, conversions):
AttributeError: module 'os' has no attribute 'EX_SOFTWARE'
The text was updated successfully, but these errors were encountered: