Please help me to rescue my deconz network, I've tried everything

Hello,
My zigbee network is getting worse and worse and I cant’ understand why.
Here is my situation:

  • I live in an apartment of around 90 square meters,
  • My conbee 2 is connected to a raspberry pi4 8gb with ssd running Home assistant,
  • I have 62 nodes (aqara, lightify, sonoff, frient and tuya zigbee)
  • My router is set to channel 1 and the zigbee network to 25 (the neighbours wifi goes from 3 to 11),
  • I’ve tried various lenght of usb extender
  • Both ssd and deconz are connected to USB 2 and there’s nothing on USB 3
  • Conbee II is in the center of the house and re wifi router is in another room about 3 meters distant (and 2 walls)
  • I’ve tried moving the conbee II in various place and tried to install phoscon (the image on the website) in a raspberry 3 b+ and place it in other places but nothing changed
  • I can’t disable wifi 2.4ghz since there are wifi device that uses it
  • everything is updated to the most recent version

The network became unresponsive the most frequent issue is door/window sensor not updatig on state change (I also tried to replace all the sonoff sensors with aqara ones) and sometimes also router devices like switches and bulbs. I constantly need to repair the connection on phoscon.

the debug log is filled with errors:

08:28:26:520 0xA4C138D5E4E8D375 error APSDE-DATA.confirm: 0xE9 on task
08:28:29:736 ZCL attribute report 0x7CB03EAA0A0AFA59 for cluster: 0x0006, ep: 0x03, frame control: 0x18, mfcode: 0x0000
08:28:37:742 0x84FD27FFFE68DF09 force poll (2)
08:28:37:758 ZCL attribute report 0x84FD27FFFE68DF09 for cluster: 0x0006, ep: 0x01, frame control: 0x18, mfcode: 0x0000
08:28:42:239 0xA4C13861CEFDC9AF error APSDE-DATA.confirm: 0xE9 on task
08:28:50:092 ZCL attribute report 0x7CB03EAA00A961C9 for cluster: 0x0006, ep: 0x03, frame control: 0x18, mfcode: 0x0000
08:28:50:113 ZCL attribute report 0x7CB03EAA00A961C9 for cluster: 0x0008, ep: 0x03, frame control: 0x18, mfcode: 0x0000
08:28:50:129 ZCL attribute report 0x7CB03EAA00A961C9 for cluster: 0x0300, ep: 0x03, frame control: 0x18, mfcode: 0x0000
08:28:50:137 ZCL attribute report 0x7CB03EAA00A961C9 for cluster: 0x0300, ep: 0x03, frame control: 0x18, mfcode: 0x0000
08:28:50:726 reuse dead link (dead link container size now 5)
08:28:52:350 ZCL attribute report 0xF0D1B80000188FFA for cluster: 0x0300, ep: 0x01, frame control: 0x08, mfcode: 0x0000
08:28:57:041 ZCL attribute report 0x84FD27FFFE3B18AF for cluster: 0x0006, ep: 0x01, frame control: 0x08, mfcode: 0x0000
08:29:07:537 0xA4C138D5E4E8D375 error APSDE-DATA.confirm: 0xE9 on task
08:29:11:718 Device TTL 2939 s flags: 0x7
08:29:13:239 ZCL attribute report 0xF0D1B80000189AFA for cluster: 0x0006, ep: 0x01, frame control: 0x08, mfcode: 0x0000
08:29:13:277 ZCL attribute report 0xF0D1B80000189AFA for cluster: 0x0008, ep: 0x01, frame control: 0x08, mfcode: 0x0000
08:29:22:270 ZCL attribute report 0x00124B0025006370 for cluster: 0x0402, ep: 0x01, frame control: 0x18, mfcode: 0x0000
08:29:25:021 ZCL attribute report 0x842E14FFFE5DB280 for cluster: 0x0006, ep: 0x01, frame control: 0x08, mfcode: 0x0000
08:29:25:685 ZCL attribute report 0x0015BC001B021CD0 for cluster: 0x0702, ep: 0x02, frame control: 0x18, mfcode: 0x0000
08:29:32:737 0xA4C13861CEFDC9AF error APSDE-DATA.confirm: 0xE9 on task
08:29:32:879 ZCL attribute report 0x00158D000836D5BB for cluster: 0x0000, ep: 0x01, frame control: 0x1C, mfcode: 0x115F
08:29:34:720 0x7CB03EAA00AA45D9 error APSDE-DATA.confirm: 0xE9 on task
08:29:36:269 ZCL attribute report 0xF0D1B80000192241 for cluster: 0x0300, ep: 0x01, frame control: 0x08, mfcode: 0x0000
08:29:45:736 ZCL attribute report 0x0015BC001B021CD0 for cluster: 0x0702, ep: 0x02, frame control: 0x18, mfcode: 0x0000
08:30:33:487 unhandled APS-DATA.confirm id: 58 status 0xE1
08:30:34:559 unhandled APS-DATA.confirm id: 65 status 0xE1
08:31:26:444 unhandled APS-DATA.confirm id: 132 status 0xE1
08:34:17:639 unhandled APS-DATA.confirm id: 54 status 0xE1
08:34:33:430 unhandled APS-DATA.confirm id: 151 status 0xE1
08:37:34:528 unhandled APS-DATA.confirm id: 166 status 0xE1
08:39:33:517 unhandled APS-DATA.confirm id: 59 status 0xE1
08:40:17:613 unhandled APS-DATA.confirm id: 38 status 0xE1
08:41:26:505 unhandled APS-DATA.confirm id: 182 status 0xE1
08:42:17:654 unhandled APS-DATA.confirm id: 201 status 0xE1
08:43:32:272 unhandled APS-DATA.confirm id: 103 status 0xE1
08:46:34:692 unhandled APS-DATA.confirm id: 150 status 0xE1
08:48:33:564 unhandled APS-DATA.confirm id: 42 status 0xE1
08:50:26:571 unhandled APS-DATA.confirm id: 201 status 0xE1
08:53:32:343 unhandled APS-DATA.confirm id: 236 status 0xE1
08:54:26:416 unhandled APS-DATA.confirm id: 18 status 0xE1
08:55:17:560 unhandled APS-DATA.confirm id: 28 status 0xE1

Here is my network:

Is there someone that has any suggestion plaease? I’m really desperate

When did this happen? Did it work stable before?

The 0xE1 is the best indicator.

Is the SSD close to the Conbee? Or in the case of the RPI?

If possible, try to move it away.

Light and bulbs were pretty stable since a month ago but battery devices always gave problems. It seems that is getting worse as I add more sensors (router or end device).
The ssd is on a external enclosure with a 15cm USB 2.0 type c cable near the raspberry but conbee II is a least 1m/1.50m away with a USB extender.
The raspberry pi 4 is connected through wifi 5.0 but I’ve tried also using lan when tried to install phoscon/deconz on the raspberry 3…

Try to move the SSD away a bit more.

Is the SSD and COnbee both plugged in USB 2.0?

I can try to use a longer type C cable for the SSD but when I installed Phoscon/Deconz on the pi3 B+ I was using the SD card slot and the log was very similar.
The pi3 B+ was placed far from the pi4 with the SSD so I don’t think that the SSD was causing interfereces on the pi3 too (I also disabled bluetooth on that installation).

Both SSD and Conbee II are connected to the USB 2.0 ports and nothing is connected to the USB 3.0.

Does it worth to try to use a powered USB hub for both SSD and Conbee II or it will slow both Deconz and Home Assistant ?

I think the SSD + WIFI usage might cause the most issues here together.

Do you have the RPI3 still around? I suggest installing raspbian on the RPI 3 with SD and follow ConBee II Installation and use Conbee 2 with that and use a LAN cable.

You can still connect it with Home Assistant :slight_smile:

That is what I already did and reported above.
I already made a clean istallation on a raspberry pi3 using the image provided on the official website and restored the backup made on the Home Assistant container.
The pi3 was connected trough LAN and I was using the SD card slot instead of the SSD but the log was filled with 0xE1 anyway.
Since I’m using the last channel avaiable for zigbee (I’m using 25 but i see that on Deconz there is 26) I don’t think that is worth trying to change the zigbee channel.
There are a lot of wifi routers on the sourrounding apartments but I’m pretty sure no one else have a zigbee network.

Did you use the extension cable on the Pi3 too?

Also, you used the LAN right? You disabled Wifi on the PI3?

I think the channel is good as it can be after checking : ZigBee and Wi-Fi Coexistence | MetaGeek

Wifi and Zigbee have different channel numbers for the freqs.

I used the lan connection without connecting the wifi, used a 2m extension cable and disabled bluetooth. the Pi3 was in the same room of the router but the conbee was at least 1,5m away from the router (but the router is using channel 1 for 2.4ghz and 116 for 5.0ghz, it’s a FritzBox 7590AX)

@de_employees Do you have any ideas?

I don’t know if it can help but this is the 2.4 wifi spectrum in my apartment:

My network is Mimi

For me he have tried all is possible to try with hardware (Raspi 3, Sd card, USB 2.0, clean install, disabled BT and wifi, …)

It’s possible a faulty device can provoke that ? The transmission issue is from the device to the network ?

Have you check if sensors with issue don’t use the same router ?

I didn’t noticed if there is a router that fails more than others but the errors are from all over the network, sometimes also routers connected directly to the conbee (and a green link on deconz) become unresponsive.
The conbee II that I’m using was updated the first time from home assistant and softbrick during the update. After several tries I was able to correctly flash the firmware again from windows and normally updated with no issue since then (always from windows).
I don’t know if it can help but I’m using this extension chord for the conbee II (1.5m) :

For the ssd I’ve tried to use my OnePlus usb cable and to insert the ssd enclosure in a shielded bag.

Can’t say if the frequency of errors is decreased (maybe it a coincidence) or not but they’re still present:

16:34:17:916 unhandled APS-DATA.confirm id: 3 status 0xE1
16:34:26:766 unhandled APS-DATA.confirm id: 51 status 0xE1
16:37:17:933 unhandled APS-DATA.confirm id: 243 status 0xE1
16:38:17:838 unhandled APS-DATA.confirm id: 46 status 0xE1
16:42:33:818 unhandled APS-DATA.confirm id: 180 status 0xE1
16:44:17:871 unhandled APS-DATA.confirm id: 238 status 0xE1
16:52:17:919 unhandled APS-DATA.confirm id: 186 status 0xE1
16:54:32:725 unhandled APS-DATA.confirm id: 154 status 0xE1
17:00:26:666 unhandled APS-DATA.confirm id: 89 status 0xE1
17:00:47:789 void zmMaster::handleStateStatus(zmMaster::MasterEvent) EVENT_TIMEOUT
17:00:47:803 void zmMaster::handleStateIdle(zmMaster::MasterEvent) unhandled event EVENT_GOT_STATUS
17:01:33:746 unhandled APS-DATA.confirm id: 148 status 0xE1
17:02:33:908 unhandled APS-DATA.confirm id: 42 status 0xE1
17:06:17:931 unhandled APS-DATA.confirm id: 211 status 0xE1
17:12:26:811 unhandled APS-DATA.confirm id: 27 status 0xE1
17:14:26:659 unhandled APS-DATA.confirm id: 199 status 0xE1
17:15:26:681 unhandled APS-DATA.confirm id: 49 status 0xE1
17:16:26:729 unhandled APS-DATA.confirm id: 145 status 0xE1
17:18:26:661 unhandled APS-DATA.confirm id: 63 status 0xE1
17:19:26:775 unhandled APS-DATA.confirm id: 169 status 0xE1
17:21:17:855 unhandled APS-DATA.confirm id: 41 status 0xE1
17:24:27:124 unhandled APS-DATA.confirm id: 22 status 0xE1
17:24:33:763 unhandled APS-DATA.confirm id: 62 status 0xE1
17:25:26:684 unhandled APS-DATA.confirm id: 115 status 0xE1
17:28:13:152 aps request id: 122 prf: 0x0104 cl: 0x0006 timeout NOT confirmed to 0x00158D00068A5304 (0x046C)
17:28:46:024 aps request id: 22 prf: 0x0104 cl: 0x0006 timeout NOT confirmed to 0xA4C13861CEFDC9AF (0x61BB)
17:29:16:425 aps request id: 118 prf: 0x0104 cl: 0x000A timeout NOT confirmed to 0xA4C13861CEFDC9AF (0x61BB)
17:30:02:863 aps request id: 130 prf: 0x0104 cl: 0x000A timeout NOT confirmed to 0xA4C138DC630C29D7 (0x18E2)
17:30:03:808 aps request id: 136 prf: 0x0104 cl: 0x000A timeout NOT confirmed to 0xA4C138B3B2B832B8 (0xB735)
17:30:09:569 aps request id: 163 prf: 0x0104 cl: 0x0300 timeout NOT confirmed to 0xF0D1B8000018F388 (0xA02D)
17:30:26:769 unhandled APS-DATA.confirm id: 160 status 0xE1
17:30:33:895 unhandled APS-DATA.confirm id: 203 status 0xE1
17:34:32:674 unhandled APS-DATA.confirm id: 55 status 0xE1
17:35:26:812 unhandled APS-DATA.confirm id: 119 status 0xE1
17:37:26:812 unhandled APS-DATA.confirm id: 3 status 0xE1
17:39:26:750 unhandled APS-DATA.confirm id: 167 status 0xE1
17:46:17:966 unhandled APS-DATA.confirm id: 213 status 0xE1
17:50:32:765 unhandled APS-DATA.confirm id: 185 status 0xE1
17:55:17:972 unhandled APS-DATA.confirm id: 83 status 0xE1
17:55:32:825 unhandled APS-DATA.confirm id: 168 status 0xE1
17:57:17:955 unhandled APS-DATA.confirm id: 227 status 0xE1
18:11:26:753 unhandled APS-DATA.confirm id: 147 status 0xE1
18:12:33:870 unhandled APS-DATA.confirm id: 253 status 0xE1

No it can’t, according to Manuel. I disagree, because I also have 0xE1. Let me show you on discord.

Ok so It seem 0xE1 is only for perturbations with the coordinator, so we can forget all others routers.

BTW have you tried to re-use an older firmware, you remember wihch one you have at start, when you don’t have thoses issues ? If you disable “source routing” ?

I don’t remember which was the first few I used, may be the one before the actual one (I’m using deconz since around November).
I switched to deconz after giving up with homey pro. The network always showed some issue but a lot less then on homey.

I don’t think that the increase of issues is related to a few updates (at least not entirely).
Unfortunately I had to increase the devices number and increasing the devices I increased the frequency of errors.
I also tried to drop off the network some of the sonoff sensors since a lot of people is complaining some issues.

I didn’t try disabling the source routing since a switch was always offline since I manually routed it through some routers.

What about adding more or better router devices? will it improve the situation?

I know it’s not a common practice and often can make thing worse, but what about splitting the network in half and using 2 conbee II devices as coordinators one in each side of the network instead of one in the center of the house?
The vast majority of the sensors is located in 2 rooms and the othe rooms have only a couple of switches and window sensors devices so I can place there each conbee II…

IDK, but it seem the error 0xE1 concern only the coordinator.
Using 2 zigbee network is not a good idea for me, you decrease the mesh and increase perturbations.

If someone with more knowledge have an idea ?

BTW what is your firmware version ATM ? I m seing there is a beta one.

Actually I’ve this versions:

Conbee II
Version 2.17.01 / 1/7/2022
Firmware 26780700

Deconz (HA addon) 6.15.0

If I’m not wrong I can’t configure a second conbee II as router (and maybe it’s not different from another router like a smart plug or the Phoscon FLS-CT that should use the same chip as deconz II)

The 0xE1 are happening directly at the location of the ConBee, meaning this status code shows only up when the firmware tries to send a command to the channel but it is too occupied to get anything on the air.

So additional routers unfortunately won’t help here. I guess it could be the SSD, even with the USB extension cable the interference can wobble through affecting near by devices. A test to verify this could be a throw away Raspbian setup running from SD-Card without the SSD.

In my test setup I have not only the ConBee on the USB extension but also the SSD and the cables are some more expensive ones with better shielding. But this can’t be said in general as USB cables and also SSD drives are wildly different in terms of interference, some are better than others.

In regards of the 0xE1 there might be some firmware mitigation possible, the thing is: the ConBee could just send out the command, ignoring the fact that the channel seems to be occupied. There is a hardware register to configure some threshold for this. Overall I’m not a fan of this as it’s bypasses the listen-before-talk sanity (being a good neighbor in the radio frequency band) but it could be helpful in situations when where are continuous 0xE1 and nothing seems to go out. I have this on my todo list for testing but no eta nor estimate if it actually works.