deploy: 9af473895b174554f25d148115bf7c9ff0ded95c

blog/2023/05/22/using-aks-and-socks-to-connect-to-a-private-azure-db/index.html

@@ -23,4 +23,4 @@ If this sounds more appealing, install <a href=https://github.com/antitree/krew-
 </span></span><span style=display:flex><span>using: image=serjs/go-socks5-proxy
 </span></span><span style=display:flex><span>Creating SOCKS5 Proxy (Pod)...
 </span></span><span style=display:flex><span>pod/davegallant-proxy created
-</span></span></code></pre></div><p>With the above proxy connection open, it is possible to access both the DNS and private IPs accessible within the k8s cluster. In this case, I am able to access the private database, since there is network connectivity between the k8s cluster and the database.</p><h2 id=caveats-and-conclusion>Caveats and Conclusion<a href=#caveats-and-conclusion class=post-heading__anchor aria-hidden=true>#</a></h2><p>The above outlined solution makes some assumptions:</p><ul><li>there is a k8s cluster</li><li>the k8s cluster has network connectivity to the desired private database</li></ul><p>If these stars align, than this solution might work as a stopgap for accessing a private Azure DB (and I&rsquo;m assuming this could work similarly on AWS).</p><p>It would be nice if Azure provided tooling similar to cloud-sql-proxy, so that using private databases would be more of a convenient experience.</p><p>One other thing to note is that some clients (such as <a href=https://dbeaver.io/ class=link--external target=_blank rel=noreferrer>dbeaver</a>) <a href=https://github.com/dbeaver/dbeaver/issues/872 class=link--external target=_blank rel=noreferrer>do not provide DNS resolution over SOCKS</a>. So in this case, you won&rsquo;t be able to use DNS as if you were inside the cluster, but instead have to rely on knowing private ip addresses.</p></div><script type=text/javascript src=https://storage.ko-fi.com/cdn/widget/Widget_2.js></script><script type=text/javascript>kofiwidget2.init("Buy me a coffee","#458588","F1F2S4LWI"),kofiwidget2.draw()</script><section id=comments class=comments><div class='container sep-before'><div class=comments><script>var getTheme=window.localStorage&&window.localStorage.getItem("theme"),getTheme=getTheme??(window.matchMedia&&window.matchMedia("(prefers-color-scheme: light)").matches?"light":"dark"),getTheme=getTheme??"dark";let theme=getTheme==="dark"?"gruvbox-dark":"github-light",s=document.createElement("script");s.src="https://utteranc.es/client.js",s.setAttribute("repo","davegallant/davegallant.github.io"),s.setAttribute("issue-term","pathname"),s.setAttribute("theme",theme),s.setAttribute("crossorigin","anonymous"),s.setAttribute("async",""),document.querySelector("div.comments").innerHTML="",document.querySelector("div.comments").appendChild(s)</script></div></div></section></article></div><div class=sidebar></div></main><footer><div class=copyright>Dave Gallant</div></footer><script src=/js/main.4be06c129d6a89e60a661c6ac8c8e0434d58fb0fa2f685f85e2c306aca62adc5e77e7c63cb1c8a2cc5794ea42927281cf868514bcdce21ddf23dc3520e6743e7.js></script><script src=/js/flexsearch.1f92282f201926136fc931aab28494815d66c4f192c6a26626bcbb08ca96473993fd64d8e0da5db39a339acd74ef453e961cae0823b3a39b9559b3670e853c6b.js></script><script defer src=https://static.cloudflareinsights.com/beacon.min.js data-cf-beacon='{"token": "b96799f53f9940dca6f660e6052ba009"}'></script></div></body></html>
+</span></span></code></pre></div><p>With the above proxy connection open, it is possible to access both the DNS and private IPs accessible within the k8s cluster. In this case, I am able to access the private database, since there is network connectivity between the k8s cluster and the database.</p><h2 id=caveats-and-conclusion>Caveats and Conclusion<a href=#caveats-and-conclusion class=post-heading__anchor aria-hidden=true>#</a></h2><p>The above outlined solution makes some assumptions:</p><ul><li>there is a k8s cluster</li><li>the k8s cluster has network connectivity to the desired private database</li></ul><p>If these stars align, than this solution might work as a stopgap for accessing a private Azure DB (and I&rsquo;m assuming this could work similarly on AWS).</p><p>It would be nice if Azure provided tooling similar to cloud-sql-proxy, so that using private databases would be more of a convenient experience.</p><p>One other thing to note is that some clients (such as <a href=https://dbeaver.io/ class=link--external target=_blank rel=noreferrer>dbeaver</a>) <a href=https://github.com/dbeaver/dbeaver/issues/872 class=link--external target=_blank rel=noreferrer>do not provide DNS resolution over SOCKS</a>. So in this case, you won&rsquo;t be able to use DNS as if you were inside the cluster, but instead have to rely on knowing private ip addresses.</p></div><script type=text/javascript src=https://storage.ko-fi.com/cdn/widget/Widget_2.js></script><script type=text/javascript>kofiwidget2.init("Buy me a coffee","#458588","F1F2S4LWI"),kofiwidget2.draw()</script><section id=comments class=comments><div class='container sep-before'><div class=comments><script>var getTheme=window.localStorage&&window.localStorage.getItem("theme"),getTheme=getTheme??(window.matchMedia&&window.matchMedia("(prefers-color-scheme: light)").matches?"light":"dark"),getTheme=getTheme??"dark";let theme=getTheme==="dark"?"gruvbox-dark":"github-light",s=document.createElement("script");s.src="https://utteranc.es/client.js",s.setAttribute("repo","davegallant/davegallant.github.io"),s.setAttribute("issue-term","pathname"),s.setAttribute("theme",theme),s.setAttribute("crossorigin","anonymous"),s.setAttribute("async",""),document.querySelector("div.comments").innerHTML="",document.querySelector("div.comments").appendChild(s)</script></div></div></section></article></div><div class=sidebar></div></main><footer><div class=copyright>Dave Gallant</div></footer><script src=/js/main.4be06c129d6a89e60a661c6ac8c8e0434d58fb0fa2f685f85e2c306aca62adc5e77e7c63cb1c8a2cc5794ea42927281cf868514bcdce21ddf23dc3520e6743e7.js></script><script src=/js/flexsearch.23514d2e85290291d7825d597a6aedb90d1c50a52d70fd8a4f2bd267ac68eea2fb6a48ac4cbd83418dbdee90421db9ac64a21af2d332eefd3ae7fd6daaf32b1c.js></script><script defer src=https://static.cloudflareinsights.com/beacon.min.js data-cf-beacon='{"token": "b96799f53f9940dca6f660e6052ba009"}'></script></div></body></html>

blog/2023/12/10/setting-up-gitea-actions-with-tailscale/index.html

@@ -72,7 +72,7 @@
 </span></span><span style=display:flex><span>          - <span style=color:#f92672>url</span>: <span style=color:#e6db74>&#34;http://gitea:3000&#34;</span>
 </span></span></code></pre></div><p>Something to consider is whether or not you want to use ssh with git. One method to get this to work with containers is to use <a href=https://docs.gitea.com/installation/install-with-docker#ssh-container-passthrough class=link--external target=_blank rel=noreferrer>ssh container passthrough</a>. I decided to keep it simple and not use ssh, since communicating over https is perfectly fine for my use case.</p><p>After adding the above configuration, running <code>docker compose up -d</code> should be enough to get an instance up and running. It will be accessible at <a href=https://gitea.my-tailnet-name.ts.net class=link--external target=_blank rel=noreferrer>https://gitea.my-tailnet-name.ts.net</a> from within the tailnet.</p><h2 id=theming>Theming<a href=#theming class=post-heading__anchor aria-hidden=true>#</a></h2><p>I discovered some nice themes for gitea <a href=https://git.sainnhe.dev/sainnhe/gitea-themes class=link--external target=_blank rel=noreferrer>here</a> and decided to try out gruvbox.</p><p>I added the theme by cloning <a href=https://git.sainnhe.dev/sainnhe/gitea-themes/raw/branch/master/dist/theme-gruvbox-auto.css class=link--external target=_blank rel=noreferrer>theme-gruvbox-auto.css</a> into <code>./data/gitea/public/assets/css</code>. I then added the following to <code>environment</code> in <code>docker-compose.yml</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-yaml data-lang=yaml><span style=display:flex><span>- <span style=color:#ae81ff>GITEA__ui__DEFAULT_THEME=gruvbox-auto</span>
 </span></span><span style=display:flex><span>- <span style=color:#ae81ff>GITEA__ui__THEMES=gruvbox-auto</span>
-</span></span></code></pre></div><p>After restarting the gitea instance, the default theme was applied.</p><h2 id=connecting-runners>Connecting runners<a href=#connecting-runners class=post-heading__anchor aria-hidden=true>#</a></h2><p>I installed the runner by <a href=https://docs.gitea.com/usage/actions/quickstart#set-up-runner class=link--external target=_blank rel=noreferrer>following the docs</a>. I opted for installing it on a separate host (another lxc container) as recommended in the docs. I used the systemd unit file to ensure that the runner comes back online after system reboots. I installed tailscale on this gitea runner as well, so that it can have the same &ldquo;networking privileges&rdquo; as the main instance.</p><p>After registering this runner and starting the daemon, the runner appeared in <code>/admin/actions/runners</code>. I added three other runners to help with parallelization.</p><p><img src=gitea-runners.png alt=image></p><h2 id=running-a-workflow>Running a workflow<a href=#running-a-workflow class=post-heading__anchor aria-hidden=true>#</a></h2><p>Now it&rsquo;s time start running some automation. I used the <a href=https://docs.gitea.com/usage/actions/quickstart#use-actions class=link--external target=_blank rel=noreferrer>demo workflow</a> as a starting point to verify that the runner is executing workflows.</p><p>After this, I wanted to make sure that some of my existing workflows could be migrated over.</p><p>The following workflow uses a matrix to run a job for several of my hosts using ansible playbooks that will do various tasks such as patching os updates and updating container images.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-yaml data-lang=yaml><span style=display:flex><span><span style=color:#f92672>name</span>: <span style=color:#ae81ff>Run ansible</span>
+</span></span></code></pre></div><p>After restarting the gitea instance, the default theme was applied.</p><h2 id=connecting-runners>Connecting runners<a href=#connecting-runners class=post-heading__anchor aria-hidden=true>#</a></h2><p>I installed the runner by <a href=https://docs.gitea.com/usage/actions/quickstart#set-up-runner class=link--external target=_blank rel=noreferrer>following the docs</a>. I opted for installing it on a separate host (another lxc container) as recommended in the docs. I used the systemd unit file to ensure that the runner comes back online after system reboots. I installed tailscale on this gitea runner as well, so that it can have the same &ldquo;networking privileges&rdquo; as the main instance.</p><p>After registering this runner and starting the daemon, the runner appeared in <code>/admin/actions/runners</code>. I added two other runners to help with parallelization.</p><p><img src=gitea-runners.png alt=image></p><h2 id=running-a-workflow>Running a workflow<a href=#running-a-workflow class=post-heading__anchor aria-hidden=true>#</a></h2><p>Now it&rsquo;s time start running some automation. I used the <a href=https://docs.gitea.com/usage/actions/quickstart#use-actions class=link--external target=_blank rel=noreferrer>demo workflow</a> as a starting point to verify that the runner is executing workflows.</p><p>After this, I wanted to make sure that some of my existing workflows could be migrated over.</p><p>The following workflow uses a matrix to run a job for several of my hosts using ansible playbooks that will do various tasks such as patching os updates and updating container images.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-yaml data-lang=yaml><span style=display:flex><span><span style=color:#f92672>name</span>: <span style=color:#ae81ff>Run ansible</span>
 </span></span><span style=display:flex><span><span style=color:#f92672>on</span>:
 </span></span><span style=display:flex><span>  <span style=color:#f92672>push</span>:
 </span></span><span style=display:flex><span>  <span style=color:#f92672>schedule</span>:
@@ -123,4 +123,4 @@
 </span></span><span style=display:flex><span>          <span style=color:#f92672>from</span>: <span style=color:#ae81ff>RFD Notify</span>
 </span></span><span style=display:flex><span>          <span style=color:#f92672>body</span>: |<span style=color:#e6db74>
 </span></span></span><span style=display:flex><span><span style=color:#e6db74>            ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_number }}</span>            
-</span></span></code></pre></div><p>And voilà:</p><p><img src=gitea-workflow.png alt=image></p><p>You may be wondering how the gitea runner is allowed to connect to the other hosts using ansible? Well, the nodes are in the same tailnet and have <a href=https://tailscale.com/tailscale-ssh class=link--external target=_blank rel=noreferrer>tailscale ssh</a> enabled.</p><h2 id=areas-for-improvement>Areas for improvement<a href=#areas-for-improvement class=post-heading__anchor aria-hidden=true>#</a></h2><p>One enhancement that I would like to see is the ability to send notifications on workflow failures. Currently, this <a href=https://github.com/go-gitea/gitea/issues/23725 class=link--external target=_blank rel=noreferrer>doesn&rsquo;t seem possible</a> without adding logic to each workflow.</p><h2 id=conclusion>Conclusion<a href=#conclusion class=post-heading__anchor aria-hidden=true>#</a></h2><p>Gitea Actions are fast and the resource footprint is minimal. My gitea instance is currently using around 250mb of memory and a small fraction of a single cpu core (and the runner is using a similar amount of resources). This is impressive since many alternatives tend to require substantially more resources. It likely helps that the codebase is largely written in go.</p><p>By combining gitea with the networking marvel that is tailscale, running workflows becomes simple and fun. Whether you are working on a team or working alone, this setup ensures that your workflows are securely accessible from anywhere with an internet connection.</p></div><script type=text/javascript src=https://storage.ko-fi.com/cdn/widget/Widget_2.js></script><script type=text/javascript>kofiwidget2.init("Buy me a coffee","#458588","F1F2S4LWI"),kofiwidget2.draw()</script><section id=comments class=comments><div class='container sep-before'><div class=comments><script>var getTheme=window.localStorage&&window.localStorage.getItem("theme"),getTheme=getTheme??(window.matchMedia&&window.matchMedia("(prefers-color-scheme: light)").matches?"light":"dark"),getTheme=getTheme??"dark";let theme=getTheme==="dark"?"gruvbox-dark":"github-light",s=document.createElement("script");s.src="https://utteranc.es/client.js",s.setAttribute("repo","davegallant/davegallant.github.io"),s.setAttribute("issue-term","pathname"),s.setAttribute("theme",theme),s.setAttribute("crossorigin","anonymous"),s.setAttribute("async",""),document.querySelector("div.comments").innerHTML="",document.querySelector("div.comments").appendChild(s)</script></div></div></section></article></div><div class=sidebar></div></main><footer><div class=copyright>Dave Gallant</div></footer><script src=/js/main.4be06c129d6a89e60a661c6ac8c8e0434d58fb0fa2f685f85e2c306aca62adc5e77e7c63cb1c8a2cc5794ea42927281cf868514bcdce21ddf23dc3520e6743e7.js></script><script src=/js/flexsearch.1f92282f201926136fc931aab28494815d66c4f192c6a26626bcbb08ca96473993fd64d8e0da5db39a339acd74ef453e961cae0823b3a39b9559b3670e853c6b.js></script><script defer src=https://static.cloudflareinsights.com/beacon.min.js data-cf-beacon='{"token": "b96799f53f9940dca6f660e6052ba009"}'></script></div></body></html>
+</span></span></code></pre></div><p>And voilà:</p><p><img src=gitea-workflow.png alt=image></p><p>You may be wondering how the gitea runner is allowed to connect to the other hosts using ansible? Well, the nodes are in the same tailnet and have <a href=https://tailscale.com/tailscale-ssh class=link--external target=_blank rel=noreferrer>tailscale ssh</a> enabled.</p><h2 id=areas-for-improvement>Areas for improvement<a href=#areas-for-improvement class=post-heading__anchor aria-hidden=true>#</a></h2><p>One enhancement that I would like to see is the ability to send notifications on workflow failures. Currently, this <a href=https://github.com/go-gitea/gitea/issues/23725 class=link--external target=_blank rel=noreferrer>doesn&rsquo;t seem possible</a> without adding logic to each workflow.</p><h2 id=conclusion>Conclusion<a href=#conclusion class=post-heading__anchor aria-hidden=true>#</a></h2><p>Gitea Actions are fast and the resource footprint is minimal. My gitea instance is currently using around 250mb of memory and a small fraction of a single cpu core (and the runner is using a similar amount of resources). This is impressive since many alternatives tend to require substantially more resources. It likely helps that the codebase is largely written in go.</p><p>By combining gitea with the networking marvel that is tailscale, running workflows becomes simple and fun. Whether you are working on a team or working alone, this setup ensures that your workflows are securely accessible from anywhere with an internet connection.</p></div><script type=text/javascript src=https://storage.ko-fi.com/cdn/widget/Widget_2.js></script><script type=text/javascript>kofiwidget2.init("Buy me a coffee","#458588","F1F2S4LWI"),kofiwidget2.draw()</script><section id=comments class=comments><div class='container sep-before'><div class=comments><script>var getTheme=window.localStorage&&window.localStorage.getItem("theme"),getTheme=getTheme??(window.matchMedia&&window.matchMedia("(prefers-color-scheme: light)").matches?"light":"dark"),getTheme=getTheme??"dark";let theme=getTheme==="dark"?"gruvbox-dark":"github-light",s=document.createElement("script");s.src="https://utteranc.es/client.js",s.setAttribute("repo","davegallant/davegallant.github.io"),s.setAttribute("issue-term","pathname"),s.setAttribute("theme",theme),s.setAttribute("crossorigin","anonymous"),s.setAttribute("async",""),document.querySelector("div.comments").innerHTML="",document.querySelector("div.comments").appendChild(s)</script></div></div></section></article></div><div class=sidebar></div></main><footer><div class=copyright>Dave Gallant</div></footer><script src=/js/main.4be06c129d6a89e60a661c6ac8c8e0434d58fb0fa2f685f85e2c306aca62adc5e77e7c63cb1c8a2cc5794ea42927281cf868514bcdce21ddf23dc3520e6743e7.js></script><script src=/js/flexsearch.23514d2e85290291d7825d597a6aedb90d1c50a52d70fd8a4f2bd267ac68eea2fb6a48ac4cbd83418dbdee90421db9ac64a21af2d332eefd3ae7fd6daaf32b1c.js></script><script defer src=https://static.cloudflareinsights.com/beacon.min.js data-cf-beacon='{"token": "b96799f53f9940dca6f660e6052ba009"}'></script></div></body></html>