• El entrenamiento provoca que la memoria RAM se desborde y el sistema operativo mata el procesor de entrenamiento → KILLED • SOLUCION: cambiar los parámetros de la configuration.py ó REDUCIR EL NUMERO DE PACIENTES SUBJECTS
– reduzco en HGG a 20 subjects y en LGG a 10
2.2.2 nombre del argumento
TypeError: get_data format() got an unexpected keyword argument 'tfmode'
python3 from tensorpack.utils.argtools import get_data_format data_format=channels_last get_data_format(data_format,tfmode=False)
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: get_data_format() got an unexpected keyword argument tfmode dir() → list import modules import inspect → https://docs.python.org/3/library/inspect.html inspect.getsource(get_data_ → captura el código del módulo get_data_format def get_data_format(data_format, keras_mode=True):\n if keras_mode:\n dic = {’NCHW\: ’channels_first’, ’NHWC’: ’chan- nels_last’}n else:\n dic = {’channels_first’: ’NCHW’, ’channels_last’: ’NHWC’}n ret = dic.get(data_format, data_format)\n if ret not in dic.values():\n raise ValueError("Unknown data_format: {}".format(data_format))\n return ret\n’ EL ERROR ES EL NOMBRE DEL ARGUMENTO: en el prototipo de la función el segundo argumento es "keras_mode" y en la llamada a la función "tfmode" Dónde está definida la función get_data_format get_data_format.globals[file] → /home/upf/.local/lib/python3.6/site-packages/tensorpack EDICION: en el fichero custom_ops.py en la llamada get_data_format(data_format,tfmode=False) cambio el argumento tfmode por keras_mode
2.2.3 Solucionado
Con los cambios realizados se produce el entrenamiento
3 RESULTADO DEL TRAINING
• Se han utilizado en HGG 20 subjects y en LGG a 10: los subjects BraTS19_2013_*
• La información sobre el training está en train_log/unet3d/log.log • upf@fpgalab:~/3DUnet-Tensorflow-Brats18$ cat train_log/unet3d/log.log [0525 06:11:27 @logger.py:92] Argv: train.py --logdir=./train_log/unet3d --gpu 0 [0525 06:11:27 @gpu.py:43] WRN Found non-empty CUDA_VISIBLE_DEVICES. But TensorFlow was ←- not built with CUDA support and could not use GPUs! [0525 06:11:50 @parallel.py:340] [MultiProcessRunnerZMQ] Will fork a dataflow more than ←- one times. This assumes the datapoints are i.i.d. [0525 06:11:50 @gpu.py:43] WRN Found non-empty CUDA_VISIBLE_DEVICES. But TensorFlow was ←- not built with CUDA support and could not use GPUs! Deep Learning: 3DUnet 3/5
[0525 06:11:50 @input_source.py:221] Setting up the queue 'QueueInput/input_queue' for CPU ←-
unet3d/lo_conv0_0/bias [16] 16 unet3d/lo_conv0_0/ins_norm/beta [16] 16 unet3d/lo_conv0_0/ins_norm/gamma [16] 16 unet3d/lo_conv1_0/kernel [1, 1, 1, 16, 16] 256 unet3d/lo_conv1_0/bias [16] 16 unet3d/lo_conv1_0/ins_norm/beta [16] 16 unet3d/lo_conv1_0/ins_norm/gamma [16] 16 unet3d/final/kernel [1, 1, 1, 16, 4] 64 unet3d/final/bias [4] 4 Number of trainable variables: 114 Number of parameters (elements): 8269676 Storage space needed for all trainable variables: 31.55MB [0525 06:11:56 @base.py:207] Setup callbacks graph ... [0525 06:11:56 @argtools.py:138] WRN Starting a process with 'fork' method is efficient ←- but not safe and may cause deadlock or crash.Use 'forkserver' or 'spawn' method ←- instead if you run into such issues.See https://docs.python.org/3/library/ ←- multiprocessing.html#contexts-and-start-methods on how to set them. [0525 06:11:56 @argtools.py:138] WRN "import prctl" failed! Install python-prctl so that ←- processes can be cleaned with guarantee. [0525 06:11:56 @summary.py:47] [MovingAverageSummary] 2 operations in collection ' ←- MOVING_SUMMARY_OPS' will be run with session hooks. [0525 06:11:56 @summary.py:94] Summarizing collection 'summaries' of size 4. [0525 06:11:56 @base.py:228] Creating the session ...
