tag:codewars.statuspage.io,2005:/historyCodewars Status - Incident History2024-03-29T05:23:02ZCodewarstag:codewars.statuspage.io,2005:Incident/166928582023-03-30T03:20:37Z2023-03-30T03:20:37ZNetwork Error<p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>03:20</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>03:07</var> UTC</small><br><strong>Monitoring</strong> - Recreated the cluster and monitoring.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>02:09</var> UTC</small><br><strong>Update</strong> - Debugging a DNS issue.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>01:34</var> UTC</small><br><strong>Update</strong> - Still waiting for the operation to finish.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>01:10</var> UTC</small><br><strong>Identified</strong> - The root certificate of the cluster had expired. Performing a credential rotation.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>01:04</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/111165722022-09-13T09:49:14Z2022-09-13T09:49:14ZBad Gateway Error<p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>09:49</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>09:32</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>09:15</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>08:56</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>08:55</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>07:59</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/108900132022-08-23T17:04:57Z2022-08-23T17:04:57ZNetwork Error<p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>17:04</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>17:03</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>09:33</var> UTC</small><br><strong>Update</strong> - A misconfigured automated tool is causing issues and cannot be disabled at this time.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>09:33</var> UTC</small><br><strong>Identified</strong> - A misconfigured automated tool is messing up the DNS records.</p>tag:codewars.statuspage.io,2005:Incident/108892762022-08-23T09:07:58Z2022-08-23T09:49:57ZNetwork Error<p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>09:07</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>09:05</var> UTC</small><br><strong>Monitoring</strong> - A fix was applied.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>08:39</var> UTC</small><br><strong>Update</strong> - I'm working on a fix. Code Runner should start to become usable.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>08:16</var> UTC</small><br><strong>Identified</strong> - The cause was identified.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>07:36</var> UTC</small><br><strong>Update</strong> - The Code Runner UI is failing to connect to the Code Runner server due to CORS error.<br />I'm still looking into what actually caused this suddenly because we haven't changed CORS options in a while.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>07:04</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/103666622022-06-21T07:48:55Z2022-06-21T07:48:55ZPartial Outage<p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>07:48</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>07:43</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>07:09</var> UTC</small><br><strong>Monitoring</strong> - We're affected by Cloudflare outage https://www.cloudflarestatus.com/incidents/xvs51y9qs9dj</p>tag:codewars.statuspage.io,2005:Incident/87427982021-12-02T21:42:10Z2021-12-02T21:42:10ZBackground queue has high latency<p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>21:42</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>19:25</var> UTC</small><br><strong>Monitoring</strong> - The fix was deployed. Waiting for the backed up jobs to be completed.</p><p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>19:10</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Dec <var data-var='date'> 2</var>, <var data-var='time'>16:01</var> UTC</small><br><strong>Investigating</strong> - We are looking into why our background task queue is backing up. New signups, honor, publishing and other key application features are affected.</p>tag:codewars.statuspage.io,2005:Incident/85568712021-11-19T03:46:44Z2021-11-19T03:46:44ZIncident affecting Search and Kata Editor<p><small>Nov <var data-var='date'>19</var>, <var data-var='time'>03:46</var> UTC</small><br><strong>Resolved</strong> - Resolving the incident because Codewars is usable with the fallback.</p><p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>05:49</var> UTC</small><br><strong>Update</strong> - Deployed a basic fallback search for Kata for when Elasticsearch is not available. Please note that some features are limited or missing.<br /><br />Working on a similar fallback for Collections search.</p><p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>05:06</var> UTC</small><br><strong>Update</strong> - We are still waiting on updates. Thanks for your patience.<br />Started working on adding a basic fallback search.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>23:19</var> UTC</small><br><strong>Update</strong> - We are still waiting for an update. Thanks for your patience.<br /><br />A possible workaround for basic search is to use search engines.<br />- https://www.google.com/search?q=site:www.codewars.com+inurl:javascript<br />- https://duckduckgo.com/?q=site:www.codewars.com+inurl:javascript<br /><br />Replace `javascript` with the language you want.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>20:13</var> UTC</small><br><strong>Update</strong> - Codewars was updated to fail more gracefully when Elasticsearch is not available. Kata Editor and Collections Search no longer crashes.<br /><br />Still waiting for updates.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>19:09</var> UTC</small><br><strong>Update</strong> - We are still waiting for updates.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>10:52</var> UTC</small><br><strong>Update</strong> - The case was reviewed and escalated. Waiting for update.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>09:35</var> UTC</small><br><strong>Update</strong> - Opened support case and waiting for response.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>08:35</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Nov <var data-var='date'>17</var>, <var data-var='time'>07:33</var> UTC</small><br><strong>Identified</strong> - Our hosted Elasticsearch is down. Kata/collection search page is affected. Kata editor is also affected.</p>tag:codewars.statuspage.io,2005:Incident/46441962020-07-17T22:35:33Z2020-07-17T22:35:33ZMajor Outage<p><small>Jul <var data-var='date'>17</var>, <var data-var='time'>22:35</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>17</var>, <var data-var='time'>21:43</var> UTC</small><br><strong>Monitoring</strong> - Cloudflare is back up and we're monitoring</p><p><small>Jul <var data-var='date'>17</var>, <var data-var='time'>21:42</var> UTC</small><br><strong>Identified</strong> - We're affected by Cloudflare outage https://www.cloudflarestatus.com/incidents/b888fyhbygb8</p><p><small>Jul <var data-var='date'>17</var>, <var data-var='time'>21:29</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/39556342020-04-24T19:46:35Z2020-04-24T19:46:35ZMajor Outage<p><small>Apr <var data-var='date'>24</var>, <var data-var='time'>19:46</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>24</var>, <var data-var='time'>19:26</var> UTC</small><br><strong>Monitoring</strong> - The network issue seems to be resolved.</p><p><small>Apr <var data-var='date'>24</var>, <var data-var='time'>19:00</var> UTC</small><br><strong>Identified</strong> - We're affected by https://status.redislabs.com/incidents/3tnrm8hw8wgl</p><p><small>Apr <var data-var='date'>24</var>, <var data-var='time'>18:56</var> UTC</small><br><strong>Investigating</strong> - Codewars is currently experiencing a major outage. It seems to be caused by our Redis servers not responding.</p>tag:codewars.statuspage.io,2005:Incident/30505432019-09-29T23:11:38Z2019-09-29T23:11:38ZMajor Outage<p><small>Sep <var data-var='date'>29</var>, <var data-var='time'>23:11</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>29</var>, <var data-var='time'>23:07</var> UTC</small><br><strong>Identified</strong> - Major outage due to our Redis servers being down.<br />https://status.compose.com/</p>tag:codewars.statuspage.io,2005:Incident/30151932019-09-19T18:12:30Z2019-09-19T18:12:30ZMajor Outage<p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>18:12</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>17:48</var> UTC</small><br><strong>Monitoring</strong> - Connection issue seems resolved. We're still monitoring.</p><p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>17:13</var> UTC</small><br><strong>Identified</strong> - All of our hosted Redis servers are having connectivity issues. https://status.compose.com/</p><p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>17:10</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Sep <var data-var='date'>19</var>, <var data-var='time'>17:03</var> UTC</small><br><strong>Investigating</strong> - Major outage due to failing to connect to all of the Redis servers. We're starting to investigate.</p>tag:codewars.statuspage.io,2005:Incident/29113922019-08-25T02:16:45Z2019-08-25T02:16:45ZScheduled Database Maintenance<p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>02:16</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>02:09</var> UTC</small><br><strong>Verifying</strong> - Database upgrade completed. We'll be bringing the service back soon after verifying.</p><p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>01:21</var> UTC</small><br><strong>Update</strong> - Successfully finished the database upgrade process. We're waiting for the secondaries to catch up. This shouldn't take more than an hour.</p><p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>23:18</var> UTC</small><br><strong>Update</strong> - Successfully started database upgrade process. This is estimated to take about 2 hours.</p><p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>23:00</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>19:57</var> UTC</small><br><strong>Scheduled</strong> - We will be undergoing scheduled database maintenance during this time. It is estimated to take about 2-3 hours.</p>tag:codewars.statuspage.io,2005:Incident/27588512019-07-24T23:17:57Z2019-07-24T23:17:57ZMajor Outage<p><small>Jul <var data-var='date'>24</var>, <var data-var='time'>23:17</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>24</var>, <var data-var='time'>23:11</var> UTC</small><br><strong>Investigating</strong> - Major outage caused by Redis becoming unavailable.</p>tag:codewars.statuspage.io,2005:Incident/25874262019-06-22T06:53:05Z2019-06-22T06:53:05ZMajor Outage<p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>06:53</var> UTC</small><br><strong>Resolved</strong> - The issue is resolved, thanks for your patience.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>06:35</var> UTC</small><br><strong>Monitoring</strong> - We'll continue monitoring.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>06:34</var> UTC</small><br><strong>Update</strong> - Waiting for the change to propagate. The service should start coming back live soon.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>06:14</var> UTC</small><br><strong>Update</strong> - We're affected by https://status.cloud.google.com/incident/container-engine/19007 . Upgrading to another version should resolve the issue.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>05:50</var> UTC</small><br><strong>Identified</strong> - We've identified that the outage is caused by change in DNS configuration. We're working on recovering the service.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>05:44</var> UTC</small><br><strong>Update</strong> - Looking into the cause of the major outage.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>05:34</var> UTC</small><br><strong>Investigating</strong> - Updates to server configuration might be causing partial outage.</p>tag:codewars.statuspage.io,2005:Incident/25473752019-06-14T10:31:15Z2019-06-14T10:31:15ZRunner Service Outage<p><small>Jun <var data-var='date'>14</var>, <var data-var='time'>10:31</var> UTC</small><br><strong>Resolved</strong> - The runner service is now fully functioning. The outage began around 1:10 AM PST and lasted about 2 hours.<br />Sorry for the inconvenience.</p><p><small>Jun <var data-var='date'>14</var>, <var data-var='time'>10:19</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Jun <var data-var='date'>14</var>, <var data-var='time'>10:18</var> UTC</small><br><strong>Monitoring</strong> - Configuration updated. The service should be recovering.</p><p><small>Jun <var data-var='date'>14</var>, <var data-var='time'>10:07</var> UTC</small><br><strong>Identified</strong> - Major outage caused by Redis service down</p>tag:codewars.statuspage.io,2005:Incident/24522062019-05-27T02:21:44Z2019-05-27T02:21:44ZMajor Outage<p><small>May <var data-var='date'>27</var>, <var data-var='time'>02:21</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>27</var>, <var data-var='time'>02:19</var> UTC</small><br><strong>Monitoring</strong> - We've switched to another portal and we're deploying the changes.</p><p><small>May <var data-var='date'>27</var>, <var data-var='time'>02:05</var> UTC</small><br><strong>Identified</strong> - The outage seems to be caused by failing to connect to the Redis service. We'll update as we find more information.</p><p><small>May <var data-var='date'>27</var>, <var data-var='time'>01:56</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/21193282018-12-29T06:50:13Z2018-12-29T06:50:13ZDegraded Performance<p><small>Dec <var data-var='date'>29</var>, <var data-var='time'>06:50</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>28</var>, <var data-var='time'>07:15</var> UTC</small><br><strong>Monitoring</strong> - Successfully updated the system. Current degraded performance is due to the cache warming up.</p>tag:codewars.statuspage.io,2005:Incident/21103682018-12-21T19:05:20Z2018-12-21T19:05:20ZPartial Outage and Degraded Performance<p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>19:05</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>07:53</var> UTC</small><br><strong>Monitoring</strong> - Please clear the cookie if you're still having problem accessing Codewars. Also, degraded performance is expected as the cache had to be cleared.</p><p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>07:31</var> UTC</small><br><strong>Identified</strong> - The outage is caused by failure to update the system. Please clear your cookie if you're having problem accessing Codewars.</p><p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>07:28</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/21059922018-12-19T20:58:53Z2018-12-19T20:58:53ZPartial Outage and Degraded Performance<p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>20:58</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>07:11</var> UTC</small><br><strong>Monitoring</strong> - The failing instance is now back up.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>03:18</var> UTC</small><br><strong>Identified</strong> - The outage is caused by partially failing to connect to the database.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>03:13</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>03:07</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/19602942018-10-10T07:40:55Z2018-10-10T07:40:55ZDegraded Performance<p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>07:40</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>07:40</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'>10</var>, <var data-var='time'>05:55</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/18181702018-07-17T19:59:07Z2018-07-17T19:59:07ZPartial Outage of Code Execution Service<p><small>Jul <var data-var='date'>17</var>, <var data-var='time'>19:59</var> UTC</small><br><strong>Resolved</strong> - The issue should be resolved. https://status.cloud.google.com/incident/cloud-networking/18012</p><p><small>Jul <var data-var='date'>17</var>, <var data-var='time'>19:41</var> UTC</small><br><strong>Investigating</strong> - Our code execution service is partially down due to network issues.<br />https://status.cloud.google.com/incident/cloud-networking/18012</p>tag:codewars.statuspage.io,2005:Incident/18107252018-07-13T00:59:48Z2018-07-13T00:59:48ZMajor Outage/Degraded Perfomance<p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>00:59</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>00:53</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>00:38</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>00:36</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jul <var data-var='date'>13</var>, <var data-var='time'>00:20</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/17514932018-06-02T01:18:46Z2018-06-02T01:18:46ZDatabase issue<p><small>Jun <var data-var='date'> 2</var>, <var data-var='time'>01:18</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'> 2</var>, <var data-var='time'>01:10</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jun <var data-var='date'> 2</var>, <var data-var='time'>01:08</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>20:24</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>20:23</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:codewars.statuspage.io,2005:Incident/17486512018-05-31T00:48:14Z2018-05-31T00:48:14ZRedis database failure<p><small>May <var data-var='date'>31</var>, <var data-var='time'>00:48</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>31</var>, <var data-var='time'>00:40</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:codewars.statuspage.io,2005:Incident/16433992018-03-16T18:47:12Z2018-03-16T18:47:12ZSlow background processing<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>18:47</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>22:35</var> UTC</small><br><strong>Identified</strong> - Background queues have backed up and will take some time to clear through</p>