Changes between Version 1 and Version 2 of Internal/StudentPages/Jiachen/ICN16Reviews

Timestamp:

Sep 13, 2016, 11:40:11 AM (8 years ago)

Author:

jiachen

Comment:

--

Internal/StudentPages/Jiachen/ICN16Reviews

@@ -19 +19 @@
 '''Weaknesses''': (Be brief)\\
 * The multipath solution does not work properly with the window adjustment on the consumer size.
-* The per FIB entry based multipath and highly congested link solution does not seem to be feasible or efficient.
+* The per FIB entry based multipath and highly congested link solution do not seem to be feasible or efficient.
 * The evaluation is not adequate.
 
 '''Detailed comments''':\\
  This paper proposed a congestion control mechanism in NDN.
- The basic solution comprises a consumer-driven window adjustment with AQM which decouples the congestion notification from packet loss (although the authors claim it is hop-by-hop solution).
- This solution can effectively address the efficiency and scalability issues caused by pure hop-by-hop flow-based fair queueing, buffer bloat, and etc.
- To enhance the solution, the authors proposed FIB-based multipath solution which would dynamically pick "forward percentage" ''for each FIB entry'' among all the outgoing faces.
+ The basic solution comprises a consumer-driven window adjustment with AQM which decouples the congestion notification from packet loss (although the authors claim it is a hop-by-hop solution).
+ This solution can effectively address the efficiency and scalability issues caused by pure hop-by-hop flow-based fair queuing, buffer bloat, and etc.
+ To enhance the solution, the authors proposed a FIB-based multipath solution which would dynamically pick "forward percentage" ''for each FIB entry'' among all the outgoing faces.
  To detect loss (which is now decoupled from congestion), the paper proposed a light-weight loss detection mechanism between layer 2 and layer 3.
  For the highly congested links caused by lower-layer multiplexing and/or unresponsive consumers, the solution simply disables a face to such a link and try to choose other paths.
 
- While the paper is overall well written, there exists some critical issues with design and evaluation:
-* The multipath solution is based on each FIB prefix. This might cause a set of problems. E.g., if we have a flow with prefix "/flow" which has 1 entry in FIB. A packet "/flow/seg_255" has another entry (maybe it is cached somewhere else). The forward percentage on "/flow" will not be affected on "/flow/*" and the forwarding might not follow the forwarding percentage.\\
- Another example is, if you have flows with prefixes "/flow1" and "/flow2", and they share a same set of outgoing faces on a router. However, the forwarding percentage are not shared among flows. How can it ensure fairness (across flows?) 
-* Another problem with MP solution is the dynamics between the consumer window adjustment and multipath forwarding percentages. When none of the multiple paths is saturated, the solution would shift the traffic back to the path with lowest latency. This is what might happen: 1) the consumer increases the window and saturate the paths and the paths share the traffic; 2) the consumer receives marks and reduce the window size (and reduce the receive rate, so that the traffic is shifted back to the low-delay link; 3) the consumer would repeat 1 and cause oscillation.\\
+ While the paper is overall well written, there exist some critical issues with design and evaluation:
+* The multipath solution is based on each FIB prefix. This might cause a set of problems. E.g., if we have a flow with prefix "/flow" which has 1 entry in FIB. A packet "/flow/seg_255" has another entry (maybe it is cached somewhere else). The forward percentage on "/flow" will not be affected by "/flow/*" and the forwarding might not follow the forwarding percentage.\\
+ Another example is, if you have flows with prefixes "/flow1" and "/flow2", and they share the same set of outgoing faces on a router. However, the forwarding percentage are not shared among flows. How can it ensure fairness (across flows?) 
+* Another problem with MP solution is the dynamics between the consumer window adjustment and multipath forwarding percentages. When none of the multiple paths is saturated, the solution would shift the traffic back to the path with the lowest latency. This is what might happen: 1) the consumer increases the window and saturate the paths and the paths share the traffic; 2) the consumer receives marks and reduce the window size (and reduce the receive rate, so that the traffic is shifted back to the low-delay link; 3) the consumer would repeat 1 and cause oscillation.\\
  Also, at the beginning, the solution only uses low-latency link until it is saturated. It is not efficient. The consumer would reduce the rate on seeing marks here, which would worsen the problem.
-* Fairness -- one of the most critical aspects with congestion control mechanism -- is only evaluated on a simple topology in 4.4, and it is left to the future work!
-* In the introduction (paragraph 3) and abstract, the authors mixed varying data chunk sizes and data packet sizes. While it is true that the data chunk size would change according to the applications, the data packet size can be kept same via the use of some transport protocols at the provider side. In the evaluation (4.3 - payload), the result shows that the solution cannot deal with varying packet size well, which includes overkill and high latency (maybe also unfairness??) when the packet size increases, and under-utilize the network when the packet size decreases. It is not suggested to assume varying data ''packet'' size for a transport protocol.
+* Fairness -- one of the most critical aspects of congestion control mechanism -- is only evaluated on a simple topology in 4.4, and it is left to the future work!
+* In the introduction (paragraph 3) and abstract, the authors mixed varying data chunk sizes and data packet sizes. While it is true that the data chunk size would change according to the applications, the data packet size can be kept same via the use of some transport protocols at the provider side. In the evaluation (4.3 - payload), the result shows that the solution cannot deal with varying packet size well, which includes overkill and high latency (maybe also unfairness??) when the packet size increases and under-utilize the network when the packet size decreases. It is not suggested to assume varying data ''packet'' size for a transport protocol.
 * The performance of using of !CoDel with ECN is not clear. !CoDel is designed to work with packet drop. This paper proposed a marking solution to improve the efficiency. However, the performance of !CoDel with marking is not well studied (or referenced to a more detailed study).
 * In section 2.1, "second" point, the authors claim that they mitigate this problem by making buffers large enough. How large can be seen as enough?
@@ -42 +42 @@
 * The evaluations are only carried on simple topologies (with only 1 bottleneck). Which is definitely not adequate for a congestion control paper. Here are some more detailed comments:
   * In sec. 4.2, you should show queue size and window size.
-  * In sec. 4.3, you used 100B payload size as default value. Why is it that small? Are you using the same value with other evaluations?
-  * In sec. 4.3 payload, you can see that the solution caused huge delay (to 3 seconds!). What about the fairness? That's the problem with !CoDel but not dropping packets. In the throughput, it is quite obvious that you are overkilling the traffic at around 12th second. Which means your window control does not work well with the long delays. between 7th and 12th second, the delay does not change. Why is that happening? Between 5th and 10th second, the queue size is not changing, why is that happening? The paper should describe these facts more properly.
+  * In sec. 4.3, you used 100B payload size as a default value. Why is it that small? Are you using the same value with other evaluations?
+  * In sec. 4.3 payload, you can see that the solution caused a huge delay (to 3 seconds!). What about the fairness? That's the problem with !CoDel but not dropping packets. In the throughput, it is quite obvious that you are overkilling the traffic at around 12th second. Which means your window control does not work well with the long delays. between 7th and 12th second, the delay does not change. Why is that happening? Between 5th and 10th second, the queue size is not changing, why is that happening? The paper should describe these facts more properly.
   * In sec. 4.3 Cross_opp_2, it is quite obvious that the solution under-utilize the network. It should be addressed more properly.
   * Looks like topo 4.4 is carefully picked to ensure that there is only 1 bottleneck in the network (on the provider side, p1+p2+p3=20+10+10<50 (R1-R2)). The paper should pick more complicated network for fairness and multipath evaluations. (Topo 10 has the same issue)
-  * In Fig. 11, PCON Cwnd, looks like the window size changed dramatically and the queue size are not stable. Why is this happening? Is it an indication of a bad solution?
-  * In table 1, here is the rate/avg (for fairness). It seams that the solution is less fair even compared with ICP.
+  * In Fig. 11, PCON Cwnd, looks like the window size changed dramatically and the queue size is not stable. Why is this happening? Is it an indication of a bad solution?
+  * In table 1, here is the rate/Avg (for fairness). It seems that the solution is less fair even compared with ICP.
 || ||=ICP=||=PCON=||
 ||=C1=||0.621531632||0.504285975||
@@ -101 +101 @@
 '''Confidence''': [Expert | Knowledgeable | Some Familiarity | None]\\
 '''Quality of writing''': [Excellent | Well written | Not well written | Poorly Written]\\
-'''Summary of the paper''': (Provide a 1-2 sentence summary of the work in your own words, including the contribution and technical depth)\\
+'''Summary of the paper''': \\
 
 '''Strengths''': (Be brief)\\
@@ -113 +113 @@
 '''Confidence''': Expert\\
 '''Quality of writing''': Well written\\
-'''Summary of the paper''': (Provide a 1-2 sentence summary of the work in your own words, including the contribution and technical depth)\\
+'''Summary of the paper''': \\
+  This paper proposed a hash solution for efficient forwarding engine lookup at the same time allows for longest prefix match.
+  The solution is novel, but the paper failed to cover security issue.
+  Nor is the evaluation adequate.
 
 '''Strengths''': (Be brief)\\
+* The solution allows hashing, longest prefix matching, and at the same time avoids tree structure
 
 '''Weaknesses''': (Be brief)\\
+* The paper claims that NDN names have important security and performance implications. However, the solution does not solve the security issue.
+* The evaluation failed to show the most important feature of the solution -- performance benefit (e.g., forwarding latency)
+* The evaluation did not compare the proposed solution with other solutions.
 
 '''Detailed comments''':\\
+  For a name with segments /N,,1,,/N,,2,,/N,,3,,/.../N,,x,,, this paper gives each prefix a hash:\\
+  __N,,1,,__ = T(/N,,1,,)\\
+  __N,,2,,__ = T(/N,,1,,/N,,2,,)\\
+  ...\\
+  __N,,x,,__ = T(/N,,1,,/N,,2,,/.../N,,x,,)\\
+  T(.) is a hash function.
 
+  To query such a content with /N,,1,,/N,,2,,/N,,3,,/.../N,,x,,, the '''consumer''' would first hash each __N,,i,,__, and carry all the __N,,i,,__s in the packet instead of the original name. On the '''provider''' side, a reverse mapping table should be created to turn the hashes back to the content. On the '''routers''', it can still perform longest prefix matching using __N,,x,,__, __N,,x-1,,__, ... until a mapping is found. This solution simiplifies the data structure in the routers (with a hashtable/BloomFilter instead of a tree/trie).
 
+ The paper claims that processing name places `non-deterministic computational burden` on routers, and the names are of variable length. The proposed solution would still result in a variable-length packet header (hash value length * name segment count) and in turn result in a non-deterministic computation burden. In the results, even with each __N,,x,,__ having 32 bits, the new length of the name is longer than the original one.
 
+ Another issue with the solution is locality. Tree structure might maintain the children of a node in a local list/hashtable so that the router can find the next level in the name hierarchy with locality. However, in the proposed solution, the location of __N,,i,,__ might be very far away from __N,,i-1,,__ which will cause CPU cache miss and cause stalling. However, this paper did not evaluate anything about lookup time.
+
+ It is not clear how the solution deals with requests like: `/filename/largest/smallest` (`largest` indicates that I need the latest version, `smallest` means I want the first segment).
+
+ It is not clear why the authors choose SHA256 as T(.). Need some motivation for these design choices.
+
+ While it is true that the paper measures the mapping time on the consumer, the improvement in the forwarding efficiency (lookup time on the routers) is not evaluated in the paper.
+
+ It is not clear how the routing is done in the network if the routers are using different T(.).
+ 
+ The paper did not compare with other solutions (e.g., GPU-based NDN router, Cisco's hardware NDN router).
+
+ Figure 3 is not visible in grayscale.
+
+ The paper mentioned security issues with human-readable names, however, it failed to address the issue with the proposed solution.
+
+ Suggestion: the packet can still carry the original name. The provider can avoid the reverse table. The application can also discard Interests caused by a collision.
+