{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Tce3stUlHN0L" }, "source": [ "##### Copyright 2020 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2022-12-14T22:50:40.497246Z", "iopub.status.busy": "2022-12-14T22:50:40.496802Z", "iopub.status.idle": "2022-12-14T22:50:40.500536Z", "shell.execute_reply": "2022-12-14T22:50:40.499957Z" }, "id": "tuOe1ymfHZPu" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "qFdPvlXBOdUN" }, "source": [ "# 変数の概要" ] }, { "cell_type": "markdown", "metadata": { "id": "MfBg1C5NB3X0" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
TensorFlow.org で実行 Google Colab で実行 GitHubでソースを表示 ノートブックをダウンロード
" ] }, { "cell_type": "markdown", "metadata": { "id": "AKhB9CMxndDs" }, "source": [ "TensorFlow の**変数**は、プログラムが操作する共通の永続的な状態を表すために推奨される方法です。このガイドでは、TensorFlow で `tf.Variable` のインスタンスを作成、更新、管理する方法について説明します。\n", "\n", "変数は `tf.Variable` クラスを介して作成および追跡されます。`tf.Variable` は、そこで演算を実行して値を変更できるテンソルを表します。特定の演算ではこのテンソルの値の読み取りと変更を行うことができます。`tf.keras` などのより高度なライブラリは `tf.Variable` を使用してモデルのパラメーターを保存します。 " ] }, { "cell_type": "markdown", "metadata": { "id": "xZoJJ4vdvTrD" }, "source": [ "## セットアップ\n", "\n", "このノートブックでは、変数の配置について説明します。変数が配置されているデバイスを確認するには、この行のコメントを外します。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:40.504439Z", "iopub.status.busy": "2022-12-14T22:50:40.503898Z", "iopub.status.idle": "2022-12-14T22:50:42.407236Z", "shell.execute_reply": "2022-12-14T22:50:42.406555Z" }, "id": "7tUZJk7lDiGo" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2022-12-14 22:50:41.438311: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 22:50:41.438412: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 22:50:41.438434: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n" ] } ], "source": [ "import tensorflow as tf\n", "\n", "# Uncomment to see where your variables get placed (see below)\n", "# tf.debugging.set_log_device_placement(True)" ] }, { "cell_type": "markdown", "metadata": { "id": "vORGXDarogWm" }, "source": [ "## 変数の作成\n", "\n", "変数を作成するには、初期値を指定します。`tf.Variable` は、初期化の値と同じ `dtype` を持ちます。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:42.411833Z", "iopub.status.busy": "2022-12-14T22:50:42.411090Z", "iopub.status.idle": "2022-12-14T22:50:45.765302Z", "shell.execute_reply": "2022-12-14T22:50:45.764598Z" }, "id": "dsYXSqleojj7" }, "outputs": [], "source": [ "my_tensor = tf.constant([[1.0, 2.0], [3.0, 4.0]])\n", "my_variable = tf.Variable(my_tensor)\n", "\n", "# Variables can be all kinds of types, just like tensors\n", "bool_variable = tf.Variable([False, False, False, True])\n", "complex_variable = tf.Variable([5 + 4j, 6 + 1j])" ] }, { "cell_type": "markdown", "metadata": { "id": "VQHwJ_Itoujf" }, "source": [ "変数の外観と動作はテンソルに似ており、実際にデータ構造が `tf.Tensor` で裏付けられています。テンソルのように `dtype` と形状を持ち、NumPy にエクスポートできます。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.769153Z", "iopub.status.busy": "2022-12-14T22:50:45.768537Z", "iopub.status.idle": "2022-12-14T22:50:45.775203Z", "shell.execute_reply": "2022-12-14T22:50:45.774643Z" }, "id": "GhNfPwCYpvlq" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape: (2, 2)\n", "DType: \n", "As NumPy: [[1. 2.]\n", " [3. 4.]]\n" ] } ], "source": [ "print(\"Shape: \", my_variable.shape)\n", "print(\"DType: \", my_variable.dtype)\n", "print(\"As NumPy: \", my_variable.numpy())" ] }, { "cell_type": "markdown", "metadata": { "id": "eZmSBYViqDoU" }, "source": [ "ほとんどのテンソル演算は期待どおりに変数を処理しますが、変数は変形できません。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.778502Z", "iopub.status.busy": "2022-12-14T22:50:45.777974Z", "iopub.status.idle": "2022-12-14T22:50:45.788733Z", "shell.execute_reply": "2022-12-14T22:50:45.788183Z" }, "id": "TrIaExVNp_LK" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "A variable: \n", "\n", "Viewed as a tensor: tf.Tensor(\n", "[[1. 2.]\n", " [3. 4.]], shape=(2, 2), dtype=float32)\n", "\n", "Index of highest value: tf.Tensor([1 1], shape=(2,), dtype=int64)\n", "\n", "Copying and reshaping: tf.Tensor([[1. 2. 3. 4.]], shape=(1, 4), dtype=float32)\n" ] } ], "source": [ "print(\"A variable:\", my_variable)\n", "print(\"\\nViewed as a tensor:\", tf.convert_to_tensor(my_variable))\n", "print(\"\\nIndex of highest value:\", tf.math.argmax(my_variable))\n", "\n", "# This creates a new tensor; it does not reshape the variable.\n", "print(\"\\nCopying and reshaping: \", tf.reshape(my_variable, [1,4]))" ] }, { "cell_type": "markdown", "metadata": { "id": "qbLCcG6Pc29Y" }, "source": [ "上記のように、変数はテンソルによって裏付けられています。テンソルは `tf.Variable.assign` を使用して再割り当てできます。`assign` を呼び出しても、（通常は）新しいテンソルは割り当てられません。代わりに、既存テンソルのメモリが再利用されます。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.792029Z", "iopub.status.busy": "2022-12-14T22:50:45.791413Z", "iopub.status.idle": "2022-12-14T22:50:45.799463Z", "shell.execute_reply": "2022-12-14T22:50:45.798903Z" }, "id": "yeEpO309QbB2" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ValueError: Cannot assign value to variable ' Variable:0': Shape mismatch.The variable shape (2,), and the assigned value shape (3,) are incompatible.\n" ] } ], "source": [ "a = tf.Variable([2.0, 3.0])\n", "# This will keep the same dtype, float32\n", "a.assign([1, 2]) \n", "# Not allowed as it resizes the variable: \n", "try:\n", " a.assign([1.0, 2.0, 3.0])\n", "except Exception as e:\n", " print(f\"{type(e).__name__}: {e}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "okeywjLdQ1tY" }, "source": [ "演算でテンソルのような変数を使用する場合、通常は裏付けているテンソルで演算します。\n", "\n", "既存の変数から新しい変数を作成すると、裏付けているテンソルが複製されます。2 つの変数が同じメモリを共有することはありません。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.802471Z", "iopub.status.busy": "2022-12-14T22:50:45.802206Z", "iopub.status.idle": "2022-12-14T22:50:45.812646Z", "shell.execute_reply": "2022-12-14T22:50:45.812100Z" }, "id": "2CnfGc6ucbXc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[5. 6.]\n", "[2. 3.]\n", "[7. 9.]\n", "[0. 0.]\n" ] } ], "source": [ "a = tf.Variable([2.0, 3.0])\n", "# Create b based on the value of a\n", "b = tf.Variable(a)\n", "a.assign([5, 6])\n", "\n", "# a and b are different\n", "print(a.numpy())\n", "print(b.numpy())\n", "\n", "# There are other versions of assign\n", "print(a.assign_add([2,3]).numpy()) # [7. 9.]\n", "print(a.assign_sub([7,9]).numpy()) # [0. 0.]" ] }, { "cell_type": "markdown", "metadata": { "id": "ZtzepotYUe7B" }, "source": [ "## ライフサイクル、命名、監視\n", "\n", "Python ベースの TensorFlow では、`tf.Variable` インスタンスのライフサイクルは他の Python オブジェクトと同じです。変数への参照がない場合、変数は自動的に割り当て解除されます。\n", "\n", "変数には名前を付けることもでき、変数の追跡とデバッグに役立ちます。2 つの変数に同じ名前を付けることができます。" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.815612Z", "iopub.status.busy": "2022-12-14T22:50:45.815114Z", "iopub.status.idle": "2022-12-14T22:50:45.822443Z", "shell.execute_reply": "2022-12-14T22:50:45.821796Z" }, "id": "VBFbzKj8RaPf" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[False False]\n", " [False False]], shape=(2, 2), dtype=bool)\n" ] } ], "source": [ "# Create a and b; they will have the same name but will be backed by\n", "# different tensors.\n", "a = tf.Variable(my_tensor, name=\"Mark\")\n", "# A new variable with the same name, but different value\n", "# Note that the scalar add is broadcast\n", "b = tf.Variable(my_tensor + 1, name=\"Mark\")\n", "\n", "# These are elementwise-unequal, despite having the same name\n", "print(a == b)" ] }, { "cell_type": "markdown", "metadata": { "id": "789QikItVA_E" }, "source": [ "変数名は、モデルの保存と読み込みを行う際に維持されます。デフォルトでは、モデル内の変数は一意の変数名を自動的に取得するため、必要がない限り自分で割り当てる必要はありません。\n", "\n", "変数は区別のためには重要ですが、一部の変数は区別する必要はありません。作成時に `trainable` を false に設定すると、変数の勾配をオフにすることができます。勾配を必要としない変数には、トレーニングステップカウンターなどがあります。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.825928Z", "iopub.status.busy": "2022-12-14T22:50:45.825487Z", "iopub.status.idle": "2022-12-14T22:50:45.830332Z", "shell.execute_reply": "2022-12-14T22:50:45.829786Z" }, "id": "B5Sj1DqhbZvx" }, "outputs": [], "source": [ "step_counter = tf.Variable(1, trainable=False)" ] }, { "cell_type": "markdown", "metadata": { "id": "DD_xfDLDTDNU" }, "source": [ "## 変数とテンソルの配置\n", "\n", "パフォーマンスを向上させるため、TensorFlow はテンソルと変数を `dtype` と互換性のある最速のデバイスに配置しようとします。つまり、GPU を使用できる場合はほとんどの変数が GPU に配置されることになります。\n", "\n", "ただし、この動作はオーバーライドすることができます。このスニペットでは GPU が使用できる場合でも浮動小数点数テンソルと変数を CPU に配置します。デバイスの配置ログをオンにすると（[セットアップ](#scrollTo=xZoJJ4vdvTrD)を参照）、変数が配置されている場所を確認できます。\n", "\n", "注：手動配置も機能しますが、[配置戦略](distributed_training.ipynb) を使用したほうがより手軽かつスケーラブルに計算を最適化することができます。\n", "\n", "このノートブックを GPU の有無にかかわらず異なるバックエンドで実行すると、異なるログが表示されます。*ロギングデバイスの配置は、セッション開始時にオンにする必要があります。*" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.833717Z", "iopub.status.busy": "2022-12-14T22:50:45.833253Z", "iopub.status.idle": "2022-12-14T22:50:45.859505Z", "shell.execute_reply": "2022-12-14T22:50:45.858948Z" }, "id": "2SjpD7wVUSBJ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[22. 28.]\n", " [49. 64.]], shape=(2, 2), dtype=float32)\n" ] } ], "source": [ "with tf.device('CPU:0'):\n", "\n", " # Create some tensors\n", " a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n", " b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])\n", " c = tf.matmul(a, b)\n", "\n", "print(c)" ] }, { "cell_type": "markdown", "metadata": { "id": "PXbh-p2BXKcr" }, "source": [ "あるデバイスで変数またはテンソルの場所を設定し、別のデバイスで計算を行うことができます。この処理ではデバイス間でデータをコピーする必要があるため、遅延が発生します。\n", "\n", "ただし、複数の GPU ワーカーがあっても 1 つの変数のコピーだけが必要な場合は、この処理を実行することができます。" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T22:50:45.862760Z", "iopub.status.busy": "2022-12-14T22:50:45.862326Z", "iopub.status.idle": "2022-12-14T22:50:45.871066Z", "shell.execute_reply": "2022-12-14T22:50:45.870456Z" }, "id": "dgWHN3QSfNiQ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(\n", "[[ 1. 4. 9.]\n", " [ 4. 10. 18.]], shape=(2, 3), dtype=float32)\n" ] } ], "source": [ "with tf.device('CPU:0'):\n", " a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n", " b = tf.Variable([[1.0, 2.0, 3.0]])\n", "\n", "with tf.device('GPU:0'):\n", " # Element-wise multiply\n", " k = a * b\n", "\n", "print(k)" ] }, { "cell_type": "markdown", "metadata": { "id": "fksvRaqoYfay" }, "source": [ "注意: `tf.config.set_soft_device_placement` はデフォルトでオンになっているため、このコードは GPU のないデバイスで実行する場合でも実行され、CPU で乗算ステップが発生します。\n", "\n", "分散トレーニングの詳細については、[ガイド](distributed_training.ipynb)をご覧ください。" ] }, { "cell_type": "markdown", "metadata": { "id": "SzCkWlF2S4yo" }, "source": [ "## 次のステップ\n", "\n", "変数が一般的にどのように使用されているかを理解するには、[自動分散](autodiff.ipynb)に関するガイドをご覧ください。" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "variable.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 0 }